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Executive  Summary 


In  October  2008,  as  it  embarked  on  its  ninth  year  of  implementation,  the  Alberta 
Initiative  for  School  Improvement  (AISI)  held  a  province-wide  colloquium  to  take  stock 
of  the  progress  of  AISI  to  date  and  to  help  set  directions  for  the  future.  Involving  key 
stakeholders,  members  of  AISFs  partnership,  AISI  staff  from  the  School  Improvement 
Branch  of  Alberta  Education,  and  AISI  project  leaders,  the  colloquium  established  an 
open  and  transparent  process  of  dialogue  and  reflection  about  AISFs  strengths  and 
limitations.  Colloquium  participants  discussed  small  and  large  adjustments  that  may  be 
needed  in  reshaping  AISFs  future  to  secure  the  best  possible  outcomes  for  the 
province's  schools. 

AISI  invited  to  the  colloquium  several  researchers  who  acted  as  critical  friends  for  the 
initiative.  They  participated  in  dialogue,  observed  AISI  presentations,  interacted  with 
stakeholders,  and  responded  to  a  range  of  the  extensive  documentation  on  AISI  and 
evaluations  of  AISI  that  had  been  produced  to  date.  Robert  Crocker,  formerly  of 
Memorial  University  in  Newfoundland,  contributed  his  considerable  expertise  in 
experimental  and  survey  design  as  well  as  statistical  meta-analysis  of  existing  data  sets 
to  raise  issues  regarding  the  measurement  of  AISFs  impact.  Dennis  Sumara  and  Brent 
Davis,  then  at  the  University  of  British  Columbia,  presented  their  field-leading  work  on 
complexity  theory  and  its  uses  in  education,  and  provided  initial  feedback  on  how  AISI 
may  or  may  not  be  operating  as  a  complex  system.  Andy  Hargreaves,  formerly  of  the 
Ontario  Institute  for  Studies  in  Education  and  now  at  Boston  College,  offered  some 
initial  observations  on  AISFs  architecture  as  a  change  strategy  and  its  similarity  to  and 
difference  from  other  systemic  change  strategies.  Finally,  Pasi  Sahlberg,  incoming 
Director  of  the  Centre  for  International  Cooperation  and  Mobility  in  Finland,  provided 
his  observations  from  an  international  policy  perspective. 

Following  the  colloquium,  these  contributors  were  invited  to  undertake  deeper  research 
on  the  design,  impact  and  future  of  AISI,  including  its  sustainability.  Sixteen  research 
questions  were  finalized  in  February  and  March  2009  through  a  process  of  consensus  by 
the  research  team  and  Alberta  Education.  Five  of  these  were  overarching  questions: 

1.  What  is  the  distinctive  theory-in-action  (change  architecture)  of  AISI? 

2.  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI? ) 

3.  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

4.  Would  the  values  of  AISI  continue  without  funding? 

5.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 
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These  were  supplemented  by  eleven  subsidiary  questions: 

1.  What  have  been  the  successes  ofAISI,  as  assessed  from  multiple  perspectives? 

2.  What  are  both  the  obvious  and  the  subtle  impacts  ofAISI? 

3.  How  is  AISI  lived  and  practiced  by  educators? 

4.  What  are  the  change  processes  at  play  for  administrators  and  teachers? 

5.  What  have  been  the  difficulties  and  challenges  ofAISI? 

6.  Has  AISI  encouraged  school  authorities  to  try  new  things? 

7.  Has  AISI  encouraged  those  involved  to  take  risks  and  to  be  more  innovative? 

8.  What  are  the  opportunities  to  expand  the  measures  ofAISI  projects  from  the 
vantage  point  of  complexity  theory? 

9.  What  are  the  opportunities  to  disseminate  knowledge  generated  by  AISI  by  using 
its  networks  and  complexity  thinking?  Have  these  opportunities  been  used  to 
promote  change  across  AISI  jurisdictions? 

10.  How  has  AISI  influenced  policy  developments  at  the  school,  jurisdictional,  and 
provincial  levels? 

11.  What  are  the  implications  of  the  research  findings  for  AISI  as  a  work  in 
progress? 

Andy  Hargreaves  agreed  to  serve  as  overall  project  coordinator  for  three  research  teams 
to  provide  multiple  perspectives  on  AISI.  Pasi  Sahlberg  would  be  a  critical  friend  who 
was  also  appointed  to  the  project  and  would  advise  AISI  on  its  work  in  light  of  global 
trends  in  education  and  especially  on  the  needs  of  high-skill,  high-achievement 
knowledge  societies. 

Components  of  the  Review 

The  multiple  perspectives  review  is  organized  in  three  data-based  studies  and  two 
reviews  that  relate  AISI  to  educational  reform  trends  elsewhere. 

1.  The  Quantitative  Meta-analysis 

The  first  research  study,  by  Robert  Crocker,  is  a  meta-analysis  of  existing  provincial 
data  sets  concerning  tested  achievement  results,  as  well  as  survey  data  of  satisfaction 
levels  for  parents,  students  and  teachers,  in  relation  to  the  effects  of  AISI.  This  study 


vi 


The  Learning  Mosaic 


tries  to  ascertain  whether  there  were  observable  changes  over  time  and  across  main 
AISI  themes  and  strategies,  the  extent  to  which  these  changes  could  be  attributed  to 
AISI,  and  whether  some  AISI  themes  and/or  strategies  were  more  effective  than  others 
in  relation  to  valued  outcomes.  In  addition,  this  study  evaluates  the  suitability  of 
existing  data  sets  and  data  gathering  processes  for  determining  the  impact  of  AISI  and 
makes  suggestions  for  future  improvement  in  AISI's  subsequent  funding  cycles. 

Given  that  AISI  was  designed  as  a  school  improvement  strategy  rather  than  an 
experimental  or  quasi-experimental  research  design,  it  is  not  possible  to  clearly  separate 
AISI  effects  from  other  changes  that  may  have  occurred.  However,  positive  change  has 
taken  place  over  time  and  across  AISI  projects  and  measures.  Effects  are  larger  for 
surveys  than  for  achievement  measures.  This  raises  the  possibility  of  a  Hawthorne  or 
halo  effect  in  which  individuals  attribute  value  to  an  intervention  not  because  of  the 
inherent  value  of  the  treatment  but  rather  because  of  the  added  attention  being  shown 
for  their  work — though  this  very  attention  and  involvement  is  a  significant  contribution 
to  teacher  morale. 

Traceable  gains  on  provincial  achievement  tests  are  marginal,  which  could  be  a  source 
of  concern.  On  the  other  hand,  external  factors — such  as  rising  immigration  trends  in 
Alberta,  an  influx  of  new  teachers  prepared  in  other  provinces,  a  surge  in  retirement  of 
experienced  teachers,  and  especially  the  increase  in  the  percentage  of  English  language 
learners — cannot  be  separated  from  AISI  outcomes  in  terms  of  the  available  data  base 
and  research  design.  Indeed,  the  lack  of  a  decline  in  achievement  scores  related  to  AISI 
in  this  context  could  be  regarded  positively. 

In  general,  despite  the  most  rigorous  efforts,  it  is  exceedingly  difficult  to  isolate  the 
independent  effects  of  AISI  using  retrospective  quantitative  data.  This  is  because 

•  only  some  AISI  projects  are  designed  to  have  a  deliberate  impact  on  tested 
student  achievement; 

•  AISI  involves  almost  all  Alberta  schools  and  has  been  increasingly  integrated 
with  other  initiatives  and  developments  for  a  decade; 

•  Existing  province-wide  data  sets  are  not  amenable  to  tracing  the  impact  of 
different  AISI  projects  on  particular  students; 

•  projects  are  often  complex  and  developmental  so  that  treatments  do  not  follow 
experimental  design  principles  with  consistent  replication  from  one  site  to 
another  or  one  year  to  the  next;  and 

•  the  data  collected  by  teachers  within  projects  such  as  provincial  achievement 
test  scores  or  satisfaction  surveys  are  sometimes  not  suitably  aligned  with  the 
purposes  of  their  projects. 
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2.  Three  Contrasting  Case  Studies  of  District  Implementation 

The  second  research  study  by  Davis  and  Sumara  analyzes  how  three  school  district 
cultures  influence  the  development  of  AISI  projects  in  their  jurisdiction. 

The  first  school  district  emphasizes  learning  as  its  central  purpose  and  is  able  to  use 
AISI  funds  to  amplify  its  pre-existing  mission  to  increase  learning  for  all  students  and 
teachers.  Most  AISI  resources  were  used  to  further  webs  of  connectivity  and 
communication  among  teachers.  Resources  are  allocated  to  releasing  teachers'  time  for 
meeting  and  coaching  in  their  schools — spreading  teacher  leadership  across  the 
profession  and  enabling  teacher  leaders  to  stay  close  to  classroom  practice.  The 
district's  decentralized  network  structure  binds  it  together  through  frequency  of 
interaction,  learning  and  change,  across  schools.  There  is  no  insistence  on  direct  control 
at  every  point  by  the  district  administration.  One  result  is  strong  personal  relationships, 
high  trust  and  intense  professional  learning  across  the  district. 

The  other  two  districts  are  less  successful  in  establishing  a  learning  culture  for  students 
and  teachers.  The  second  district  promotes  service  as  the  work  for  the  district. 
Educators  work  extremely  hard,  they  are  bound  by  a  common  moral  purpose,  and 
personal  sacrifice  of  time  and  energy  is  a  pervasive  virtue.  However,  personal  relations 
and  lines  of  authority  operate  in  a  largely  vertical  fashion  and  schools  are  not  well 
connected  to  each  other  except  through  common  training  and  implementation  events. 
Teacher  leadership  largely  takes  teachers  out  of  the  classroom  into  consultant  positions 
in  the  district  office.  This  results  in  the  district's  focus  being  something  of  a  mystery  to 
many  schools.  This  makes  it  difficult  for  the  district  to  play  an  active  role  in  producing 
new  kinds  of  knowledge. 

The  third  district  employs  a  managerial  framework  that  realizes  the  aspirations  of  the 
educational  leaders  for  control  of  teachers  and  accountability  for  outcomes.  Initiatives 
are  aligned  with  a  single  district  focus  that  is  in  turn  related  to  a  provincial  "thrust." 
Resources  are  allocated  to  employing  consultants  in  the  district  office,  and  hiring 
external  out-of-province  trainers  to  distribute  training  packages  and  run  workshop 
events.  There  is  little  independent  interaction  between  or  learning  among  schools 
independently  of  these  events  and  schools  have  little  awareness  of  each  other's 
activities.  Teachers  become  depleted  by  what  they  experience  as  excessive  auditing  of 
their  instruction  without  a  parallel  system  of  continuing  support  and  development  for 
their  teaching. 

This  study  indicates  that  external  improvement  initiatives  such  as  AISI  amplify  the  pre- 
existing mindsets  of  district  administrators.  All  three  district  sites  use  AISI  funds  to 
extend  their  already  established  principles  and  strategies.  They  do  not  use  the  funds  to 
reflect  critically  upon  goals  and  interventions  that  are  already  in  place.  If  AISI  is  to 
change  existing  district  cultures  and  their  impact  on  learning  and  teaching,  it  will  be 
necessary  to  develop  robust  new  strategies  for  organizational  learning  that  transcend 
district  boundaries.  One  possibihty  would  be  to  fund  networks  of  learning  where 
educators  from  one  district  visit  peers  in  others.  Some  of  this  interaction  should  be 
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school  to  school,  without  all  interactions  being  orchestrated  by  vertical  district  control. 
Across  districts  there  was  a  rather  remarkable  knowledge  of  contemporary  educational 
research,  and  this  phenomenon  was  consistently  attributed  to  AISI.  There  were  also 
pockets  of  remarkable  innovation  in  the  districts  visited,  with  occasional  deep 
commonalities  in  interest,  expertise,  and  activity  across  jurisdictions.  At  the  same  time, 
there  seemed  to  be  little  sustained  and  meaningful  collaboration  among  jurisdictions, 
raising  the  possibility  that  the  time  may  be  right  for  a  more  deliberate  strategy  of 
connecting  and  collaborating  within  the  initiative.  AISI  needs  to  stress  the  importance 
of  creating  new  knowledge  through  school-based  innovation  as  in  the  first  district  and 
not  just  disseminating  existing  knowledge  through  measures  such  as  workshops  and 
training  programs  as  in  the  other  two.  In  recognition  of  this  need,  AISI  leaders  have 
already  begun  catalyzing  cross-district  learning  by  requiring  project  applicants  in  Cycle 
4  to  learn  from  the  work  other  districts  have  already  undertaken  in  areas  with  affinities 
to  their  proposed  project  goals  and  strategies. 

3.  Cross-site  Case  Study 

The  third  study  by  Shirley  and  McEwen  is  a  qualitative  condensed  case  study  of  12 
varied  and  geographically  dispersed  school  districts  supported  by  AISI.  This  study 
employs  interviews  and  focus  group  discussions  as  well  as  analysis  of  school,  district 
and  project  documents,  to  gather  data  about  the  meaning  and  value  of  AISI  among 
educators  and  district  personnel  involved  in  and  responsible  for  design,  implementation, 
and  assessment  of  AISI  activities  at  the  district  level.  This  up-close  view  of  AISI 
provides  evidence  of  the  perceived  architecture,  impact,  strengths  and  challenges  of 
AISI  as  well  as  the  context  in  which  AISI  operates  among  those  who  are  most  closely 
engaged  with  it. 

The  research  findings  indicate  that  AISI  enjoys  enormous  popularity  among  educators. 
They  credit  it  with  helping  them  to  advance  their  skills  as  thinkers,  researchers,  and 
practitioners.  Teachers  state  that  they  have  acquired  new  skills  as  researchers  and 
change  agents  who  identify  problems  in  children's  learning,  collaborate  with  colleagues 
to  formulate  potential  solutions,  and  then  acquire  funding,  skills  and  support  to  put  their 
professional  knowledge  to  work.  Educators  have  side-stepped  the  kinds  of  short-term 
strategies  that  lead  to  "gaming  the  system"  to  get  test  scores  up.  Instead,  they  are  asking 
more  profound  questions  of  themselves  and  of  their  colleagues.  They  are  challenging 
each  other  to  work  with  students  to  establish  agreed-upon  criteria  for  excellence  in 
learning  and  they  are  providing  children  with  multiple  ways  to  acquire  and  demonstrate 
excellence.  They  are  embedding  technology  into  a  repertoire  of  instructional  strategies 
that  presume  intelligence  and  voice  on  the  part  of  learners. 

Alberta  educators  feel  that  their  long-term  vision  of  educational  change  matters  and  that 
it  plays  a  role  in  shaping  the  future  policies  of  their  province.  They  are  undoubtedly  at 
the  leading  edge  of  efforts  to  professionalize  teaching  internationally,  through  their 
promotion  of  collegial  interaction,  change  advocacy,  professional  networks,  sustained 
inquiry  and  responsibility  for  results. 
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This  report  finds  that  areas  in  which  AISI  needs  to  push  further  include 

•  increased  parent  and  community  engagement  in  the  focus  of  AISI  projects  and 
throughout  the  whole  AISI  process; 

•  a  stronger  focus  and  greater  impact  on  secondary  schools; 

•  creating  an  accountability  system  that  is  less  cumbersome,  yet  that  develops  and 
deploys  more  robust  indicators  of  progress  that  are  related  to  the  project  goals 
that  schools  pursue. 

Of  these  three  areas,  the  research  team  notes  that  AISI  has  increased  emphasis  on  parent 
and  community  engagement  in  Cycle  4. 

4.  The  Four  Ways  of  AISI 

Andy  Hargreaves  draws  on  the  three  above  components  of  the  review  and  other 
previous  research  studies  of  AISI  to  judge  how  AISI  compares  to  four  ways  of  system- 
wide  change  that  are  and  have  been  evident  in  international  educational  change 
strategies.  He  argues  i]\2^.four  change  imperatives  now  confront  all  educational  leaders 
and  change  agents — economic  reconstruction,  social  cohesion,  ecological  sustainability, 
and  generational  renewal.  The  challenge  for  policy  makers,  he  says,  is  to  respond  to 
these  four  imperatives  in  order  to  generate  the  changes  that  are  appropriate  for  the  21^^ 
century. 

Hargreaves  argues  that  AISI  has  gone  well  beyond  a  1970s  First  Way  of  change  that 
emphasized  innovation  and  generous  state  funding  but  failed  to  develop  parallel 
systems  of  professional  responsibility,  accountability,  and  also  consistency.  AISI  also 
has  surpassed  Second  Way  mindsets  of  the  1980s,  in  which  educators  in  other 
jurisdictions  were  subjected  to  increasingly  political  control,  public  skepticism  and 
market  competition.  By  promoting  high  levels  of  public  confidence  in  educators  and  by 
emphasizing  creativity,  complexity,  innovation,  and  teamwork,  AISI  would  appear  to 
have  institutionalized  Third  Way  principles  of  the  1990s  that  used  networks  and  data  to 
drive  reform  through  recalcitrant  systems  and  educators. 

For  many  contemporary  school  reformers,  the  development  of  such  "data-driven 
decision  making"  among  educators  represents  the  culmination  of  decades  of  efforts  to 
improve  and  secure  teaching  as  a  profession.  Yet  increasing  international  evidence 
indicates  that  such  a  framing  of  contemporary  education  injects  a  managerial  tenor  and 
competitive  framing  into  schools  that  in  many  ways  undermines  their  moral  and 
collegial  fabric.  For  this  reason,  Hargreaves  indicates  that  policy  makers  in  Alberta 
would  be  well  advised  to  consider  the  role  that  AISI  might  play  in  promoting  a  Fourth 
Way  of  change  that  values  data  along  with  teachers'  professional  judgment  and  balances 
targeted  interventions  in  children's  areas  of  academic  weaknesses  along  with  more 
mindful  approaches  to  teaching  and  learning  that  nourish  creativity,  innovation,  and  the 
"soft  skills"  of  teamwork  and  compromise. 


X 


The  Learning  Mosaic 

5.  Global  Policy  Perspectives 

Hargreaves'  challenge  to  policy  makers  to  fortify  and  extend  AlSI's  already  significant 
contribution  to  Albertan  education  is  next  complemented  by  Pasi  Sahlberg,  who 
considers  AISI  from  the  vantage  point  of  his  scholarship  on  knowledge  societies  and 
especially  high-achieving  Finland.  Sahlberg  notes  that  AISFs  change  architecture 
promotes  systems-level  change  and  not  just  a  loose  accumulation  of  localized 
initiatives.  He  praises  the  generous  and  continual  allocation  of  substantial  resources  to 
AISI  and  their  encumbered  nature,  which  prevents  them  from  being  bled  off  into  staff 
replacements  or  other  expenditures  that  are  especially  salient  in  a  time  of  economic 
contraction. 

Enjoying  high-level  provincial  leadership,  inviting  grass-roots  initiatives,  and 
encouraging  mid-level  school  district  coordination  and  learning,  AISI  is  viewed  by 
Sahlberg  as  "a  shining  star  in  the  sky  of  global  large-scale  school  improvement." 
Sahlberg  credits  AISI  leaders  and  Alberta  Education  with  the  development  of  a 
carefully  conceived  and  highly  responsive  change  network  that  is  perhaps  unique  in  its 
support  for  the  technical  core  of  teaching  and  learning  that  occurs  between  teachers  and 
students.  Sahlberg  concludes  that  "It  is  difficult  to  find  anywhere  a  comparable  change 
effort  that  would  be  of  the  scale,  size  and  overall  magnitude  as  AISI." 

Sahlberg,  like  Hargreaves,  encourages  AISI  to  consider  two  further  developments  for  its 
medium  and  long-range  planning.  First,  he  suggests  that  the  central  management  of 
AISI  expand  the  definition  of  public  engagement  in  AISI  Education  Partners  to  include 
individuals  from  youth,  sports,  or  business  sectors.  Second,  he  finds  that  although  there 
is  evidence  of  some  lateral  networking  among  AISI  projects  and  districts,  this 
dimension  of  the  work — the  "communicative  connectivity"  in  the  language  of  Davis 
and  Sumara — could  be  strengthened  and  would  benefit  AISI  in  the  future. 

Overall  Findings 

This  section  summarizes  the  findings  of  the  review.  It  is  organized  by  the  overarching 
questions. 

AISI  constitutes  a  world-class  and  world-leading  example  of  a  system-wide  educational 
strategy.  This  strategy,  designed  by  Alberta  Education  and  its  partners,  inspires  teachers 
and  administrators.  It  enhances  their  professional  growth  and  enthusiasm.  AISI  seeds 
new,  research-informed  practices  within  local  communities  then  spreads  them  across 
districts  and  schools;  and  it  diffuses  existing  knowledge  as  well  as  creating  new 
knowledge. 

AISI  embodies  a  change  process  that  addresses  the  complexity  and  adaptability 
necessary  in  a  fast-moving,  knowledge-driven  world.  It  avoids  the  excesses  of 
unregulated  chaos  and  permissiveness  of  uncoordinated  innovation  on  the  one  hand,  and 
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of  hierarchical  and  inflexibly  linear  systems  of  top-down  or  layered  implementation  on 
the  other.  It  achieves  all  this  new  and  ground-breaking  work  with  no  discernible 
negative  impact  on  the  exemplary  record  of  student  performance  as  measured  by 
provincial  achievement  tests  for  which  Alberta  has  become  world-renowned. 

AISI  has  unfolded  in  a  continuous  culture  of  inquiry,  openness,  reflection  and 
adaptation  that  is  rare  among  government-sponsored  innovations.  The  School 
Improvement  Branch  of  Alberta  Education  does  not  merely  endure  critical  feedback  but 
actively  solicits  and  then  rapidly  responds  to  it.  All  projects  have  onerous  accountability 
requirements  and  have  been  subject  to  rigorous  evaluation,  leading  to  clear 
consequences  of  adaptation,  change,  and  shifts  of  focus  or  direction.  In  the  past  decade, 
AISI  has  transformed  and  continues  to  transform 

1.  from  a  project-driven  and  initiative-driven  approach  to  a  more  embedded  and 
continuous  change  process  and  strategy; 

2.  from  a  predictable,  time-bound  planning  process  of  uniform  funding  cycles,  to  a 
more  flexible  process  of  planning  and  development; 

3.  from  a  collection  of  disconnected  or  loosely  connected  projects  to  a  province- 
wide  network  of  improvement  and  innovation; 

4.  from  a  change  process  that  has  swung  between  bottom-up  and  top-down 
orientations  in  the  first  two  cycles,  to  a  change  process  that  balances  and 
integrates  these  dynamics  and  also  adds  a  strong,  lateral  peer-driven  change 
dynamic  in  the  third  and  fourth  cycles; 

5.  from  a  strategy  to  spread  and  embed  existing  knowledge  in  order  to  enhance 
improvement  and  implementation,  to  a  strategy  that  also  creates  new  knowledge 
in  support  of  increased  innovation. 

There  are  also  some  limitations  of  AISI  so  far.  For  example,  elementary  schools  have 
embraced  AISI  more  deeply  than  high  schools,  where  teachers'  understandings  of  their 
roles  as  experts  in  the  area  of  academic  content  knowledge  have  made  it  difficult  for 
them  to  focus  on  the  province's  learners  and  their  current  and  future  needs.  AISI  also 
needs  to  work  more  deliberately  on  leadership  development,  and  especially  on 
modifying  the  roles  of  principals  and  other  staff  to  support  the  development  of  teachers. 
AISI  projects  can  also  benefit  from  more  robust  knowledge  dissemination  and  exchange 
across  district  lines.  Finally,  there  is  scope  for  more  explicit  attention  to  the 
development  of  stronger  relationships  with  parents  and  other  community  members. 
These  and  other  findings  are  organized  in  relationship  to  the  five  overarching  questions 
that  guided  this  study  and  are  elaborated  in  a  following  section  on  recommendations. 
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1.  What  is  the  distinctive  theory-in-action  (change  architecture)  ofAISI? 

AISI  promotes  and  funds  locally-developed,  district-led  innovations  and  improvements. 
It  networks  educators  and  parent  and  community  members  together  through  schools, 
conferences,  and  a  web-based  AISI  Clearinghouse.  To  do  so,  AISI  has  a  four- 
dimensional  architecture: 

1.  vertical  -  top-down  and  bottom-up; 

2.  lateral  -  project-to-project,  school-to-school; 

3.  radial  -  outside-in  and  inside-out  research  expertise  and  practitioner  inquiry; 

4.  temporal  -  connecting  medium-term  and  longer-term  perspectives. 

AISI  is  a  complex  mixture  of  top-down,  bottom-up  and  laterally-driven  change.  It  is 
guided  by  the  AISI  Education  Partners  Steering  Committee  and  managed  (but  not 
micromanaged)  by  the  School  Improvement  Branch  (SEB)  of  Alberta  Education.  SEB 
works  collaboratively  with  the  AISI  partners  to  set  priorities  and  strategic  directions  for 
each  cycle.  SEB  manages  three-year  project  cycles;  it  further  manages  the  application 
and  approval  process,  coordinates  conferences  and  updates  a  website  Clearinghouse  to 
create  connectivity  across  projects.  SEB  operates  in  a  consistently  transparent,  inclusive 
and  responsive  way,  with  a  quiet  passion  for  locally-grounded  and  professionally  driven 
change  that  serves  the  public  good  of  all  students.  It  sees  its  role  as  facilitating,  steering 
and  gently  but  firmly  monitoring  and  revising  this  process  over  time. 

From  the  bottom-up,  AISI's  theory-in-action  empowers  educators  to  develop 
professional  and  intellectual  projects  based  on  their  own  locally-created  needs 
assessments  and  subsequent  initiatives  for  self-initiated  change.  Many  of  these  projects 
come  from  the  individual  passions  or  recent  professional  development  experiences  of 
teachers  and  administrators  who  connect  their  initiatives  to  the  priorities  in  the  current 
AISI  Cycle.  Others  -  up  to  40%  per  cycle  -  are  selected  by  districts  in  relation  to 
province-wide  themes  such  as  differentiated  instruction,  professional  learning 
communities  (PLCs)  or  assessment  for  learning  that  are  AISI  priorities  and  also  related 
to  a  more  general  policy  thrust  in  Alberta  Education.  Although  all  projects  feel  local  in 
location,  many  are  nonetheless  provincially  central  in  origin.  Irrespective  of  the  source, 
what  matters  in  any  project  is  the  degree  of  ownership  teachers  and  school 
administrators  feel  towards  it. 

AISI  is  not  only  bottom-up,  top-down  and  lateral  in  nature;  it  also  is  radial,  combining 
inside-out  and  outside-in  change  processes  that  penetrate  into  its  core  and  back  out 
again.  Several  districts  have  collaborated  with  university  faculty  at  various  points  in 
their  project  cycles  and  received  assistance  in  designing  surveys,  studying  student 
achievement  data,  and  modifying  assessment  practices.  Annual  AISI  conferences  also 
connect  participating  schools  to  outside  expertise  and  feedback.  AISI  has  made  explicit 
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the  connection  between  academic  research  and  professional  practice.  External 
stimulation  and  assistance  are  balanced  and  integrated  with  internal  study  and 
reflection. 

Like  all  change  strategies,  AISI  also  has  a  fourth  dimension:  time.  AISI's  three-year 
cycles  establish  longer  timelines  for  change,  action  and  results  than  is  common  in  most 
other  system-wide  reform  efforts.  In  less  stable  political  environments,  these  are 
usually  driven  by  the  demand  for  measurable  short-term  achievement  results.  In  these 
other  cases,  this  culture  of  short-term  planning  and  thinking  can  deplete  energy  and 
distract  attention  from  securing  the  longer-term  transformations  in  teaching  and  learning 
that  are  more  appropriate  for  competitive  knowledge  economies.  AISI  largely  avoids 
these  distractions  through  an  approach  that  is  iterative,  transparent,  and  participatory. 
Project  participants  consistently  praised  AISI  staff  for  their  accessibility  and  respect  for 
the  on-the-ground  realities  of  teachers  and  school  staff. 

Some  respondents  advised  greater  fluidity  in  terms  of  entry  to  and  completion  of  AISI 
projects.  They  also  suggested  it  would  be  prudent  to  reduce  the  accountability  demands 
on  projects,  which  were  described  as  onerous.  Last,  the  case  studies  and  school  district 
reviews  indicated  that  while  interconnection  (or  connectivity)  across  schools  within 
districts  is  strong,  it  is  underdeveloped  across  districts.  Districts  also  vary  in  how  they 
articulate  school  interconnections.  On  the  whole,  though,  the  current  change 
architecture  of  AISI  enables  its  project  leaders,  in  collaboration  with  district  personnel, 
to  develop  an  approach  to  student  learning  and  staff  development  that  is  more  inquiry- 
oriented,  reflective,  and  sustainable  than  most  strategies. 

2.  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI?) 

Positive  changes  over  time  were  found  for  all  measures  in  all  three  AISI  cycles.  AISFs 
impact  on  provincial  achievement  tests  (PATs)  was  small  but  larger  for  local 
achievement  measures  and  survey  measures.  Discernible  effect  sizes  on  PATs  are 
rather  modest,  and  many  seem  attributable  to  being  statistical  artifacts  of,  for  example, 
regression  towards  the  mean  or  outlier  effects. 

PAT  results  can  be  interpreted  in  a  number  of  ways.  One  potential  explanation  is 
methodological.  PAT  data  are  not  easily  connected  to  traces  of  particular  students  who 
have  experienced  specific  AISI  initiatives.  Another  possible  explanation  is  systemic. 
AISI  has  become  increasingly  integrated  into  the  educational  system  and  improvement 
processes  of  the  province  as  a  whole.  It  is  a  complex  reform,  not  a  simple  treatment  or 
intervention,  and  part  of  its  success  is  its  increasing  influence  on  the  educational  culture 
in  general.  Highlighting  its  independent  impact  is  far  from  easy.  One  promising  step 
forward  might  be  to  design  some  AISI  projects  as  experimentally  controlled 
interventions. 

The  strongest  AISI  impacts  were  on  measures  of  teacher  growth.  It  is  possible  that  these 
represent  a  halo  or  Hawthorne  effect  although  that  in  itself  is  an  indicator  of  teachers' 
appreciation  of  the  trust,  resources  and  recognition  that  are  accorded  to  them  in  the  AISI 
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architecture.  Our  review's  quahtative  findings  suggest  something  deeper  is  also  at  work 
in  terms  of  AISFs  impact  on  teachers'  sense  of  professionaHsm  and  on  the  development 
of  teacher  leadership  opportunities  and  experiences.  These  factors  represent  AISI's 
values  as  much  as  its  actual  value. 

Informants  all  agreed  that  AISI  is  catalyzing  authentic  and  deep  conversations  about 
teaching  and  learning  that  are  contributing  to  a  richer  repertoire  of  instructional 
practices  and  improved  student  learning  in  Alberta.  They  credited  AISI  with  giving 
them  new  ways  to  observe  student  learning,  identify  obstacles  to  achievement,  and 
revise  instruction  so  that  their  students  learn  at  high  levels.  By  exposing  educators  to 
alternative  sets  of  practices,  by  embedding  ongoing  support  into  schools  through  AISI- 
funded  lead  teachers  and  consultants,  by  connecting  teachers  and  projects  to  each  other 
in  relationships  of  mutual  learning  and  support,  AISI  has  helped  to  re-ignite  teachers' 
curiosity  about  new  and  better  ways  of  teaching  their  students. 

3.  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

Educators  tended  to  view  AISI  not  so  much  as  the  point  of  departure  for  new  values,  but 
rather  as  an  opportunity  and  funding  source  to  realize  values  they  already  cherished  but 
found  difficult  to  fulfill.  Districts  needed  funding  to  support  AISI  consultants,  to 
provide  teachers  with  release  time  to  learn  from  their  colleagues,  to  purchase  resources, 
and  to  send  teachers  to  professional  development  activities  such  as  the  annual 
conferences  of  the  Alberta  Assessment  Consortium.  Especially  in  remote  rural  districts, 
the  opportunity  to  leave  small  towns  to  access  new  ideas  and  research  findings  at 
provincial  or  regional  conferences  and  establish  lateral  learning  networks  with 
educators  in  implementing  them  was  priceless.  Districts  would  almost  certainly  not 

AISI  has  also  helped  combat  conservatism  in  the  culture  of  teaching  and  administration 
by  promoting  a  culture  of  risk-taking.  In  his  remarks  at  the  AISI  Conference  in 
February  2009,  Alberta's  Minister  of  Education,  Dave  Hancock,  communicated  that 
mistakes  were  to  be  expected  and  welcome  along  the  way  to  meaningful  school  change. 
Such  encouragement  was  very  much  appreciated  by  educators  who  were  eager  to  pilot 
new  initiatives  and  to  take  greater  risks  to  reach  disengaged  students. 

Teachers  stated  that  AISI  projects  offered  just  the  right  amount  of  risk  and  reward  for 
those  who  loved  teaching  yet  also  wanted  to  explore  other  dimensions  of  the 
educational  profession.  AISI  enables  teachers  to  develop  new  skills  in  the  areas  of 
experiential  education,  technology  development,  and  local  history  that  may  not  be 
directly  linked  to  gains  on  provincial  achievement  tests  but  nonetheless  have  great 
educational  value.  This  approach  is  integral  to  the  deployment  of  21^'  century 
professional  skills  in  a  rapidly  changing,  culturally  diverse  and  knowledge-driven 
society.  It  is  essential  to  a  learning  mindset. 


XV 


The  Learning  Mosaic 

4.  Would  the  values  ofAISI  continue  without  funding? 

Many  participants  from  elementary  schools  said  the  cultures  of  their  schools  had 
changed  and  the  practices  that  came  about  due  to  AISI  were  now  embedded  in  their 
schools.  In  secondary  schools,  AISI  values  were  embedded  in  some  departments  but 
others  conserved  a  transmission  model  of  education  that  did  not  promote  student 
engagement.  Gains  are  being  made  at  the  high  school  level,  but  AISI  project  leaders 
indicated  that  improvements  require  more  careful  modeling  and  support  for  faculty  over 
time  of  the  kind  that  appears  to  be  the  case  in  elementary  schools. 

The  districts  and  cultures  most  likely  to  sustain  AISI  values  in  the  absence  of  continuing 
funding  are  those  that  already  operate  as  complex  and  effective  learning  communities. 
These  districts  have  established  the  organizational  cultures  that  support  teachers' 
continued  introspection,  collaborative  inquiry,  and  adjustment  of  instructional  practices. 
Such  districts  organize  their  leaders  and  not  just  their  teachers  into  PLCs  to  study  data 
and  research  and  to  inquire  into  and  improve  instructional  supports. 

The  Davis  and  Sumara  study  of  contrasting  district  implementation  indicates  that  some 
districts  organize  their  cultures  more  around  foci  such  as  service  and  management  that 
tend  to  concentrate  leadership  centrally  and  administer  projects  vertically  rather  than 
around  learning,  where  both  leadership  and  innovation  are  distributed  more  laterally  and 
bound  together  by  frequent,  complex  interaction.  The  former  types  of  districts  may  find 
it  difficult  to  accomplish  the  learning  goals  they  have  established  for  themselves 
because  they  conflict  with  pre-established  institutional  cultures  that  make  learning 
subsidiary  to  service  or  management.  AISI  values  can  be  piloted  in  such  organizational 
cultures,  but  they  cannot  become  embedded,  and  they  are  unlikely  to  be  sustained 
without  funding  and  also  a  development  of  networking  structures  within  AISI  that  may 
stimulate  productive  disturbance  of  these  existing  district  cultures. 

Some  educators  expressed  anxiety  that  AISI  funding  might  be  folded  into  base 
budgeting,  let  alone  discontinued  entirely,  especially  in  a  time  of  economic  contraction. 
They  worried  that  without  clearly  marked  funding,  the  sorts  of  innovative,  grass-roots 
projects  associated  with  AISI  will  fade  away.  They  feared  that  without  continued 
support  for  AISI  as  an  autonomous  agency,  their  schools  would  not  prosper  from  the 
opportunity  to  mature  into  the  more  complex  learning  systems  and  sources  of 
innovation  that  Alberta  will  need  for  its  students  to  thrive  in  the  future. 

5.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

AISI's  change  architecture  has  led  to  clear  shifts  in  the  culture  of  teaching  and 
improvement  in  Alberta.  We  found  many  instances  of  AISI  influencing  school  and 
district  policies  in  ways  that  represented  a  marked  shift  in  understandings  about 
teaching  and  learning  at  the  school  and  classroom  level.  This  was  evidenced  in  the 
creation  of  common  report  cards,  the  alignment  of  curricular  content  with  local 
assessments,  and  the  development  of  principals  as  instructional  leaders  of  learning,  for 
example. 
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Instead  of  seeing  assessments  as  unwanted  external  impositions  of  provincial 
achievement  tests,  the  emphasis  on  assessment  for  learning  has  helped  teachers  grasp 
the  value  of  diagnostic  and  formative  assessments  that  can  support  their  classroom 
practice.  There  is  room  for  further  growth  in  terms  of  teachers  and  schools  developing 
and  deploying  more  of  their  own  designed  or  chosen  quantitative  instruments  and 
indicators  so  they  can  monitor  impact  of  and  progress  in  their  self-designed  initiatives; 
but  the  emphasis  on  assessment  for  learning  in  AISFs  third  cycle  has  undoubtedly 
started  to  lay  a  foundation  of  a  learning-driven  culture  of  greater  assessment  literacy. 

One  clear  and  demonstrable  impact  of  AISI  on  the  wider  educational  culture  of  Alberta 
is  in  terms  of  consolidating  and  extending  a  strong  and  enthusiastic  culture  of 
professionalism  and  professional  collaboration  among  schools  and  their  teachers. 
Without  exception,  all  of  the  educators  and  parents  we  interviewed  were  enthusiastic 
about  AISI,  and  the  way  that  it  energized  the  profession.  Schools  have  changed  as  a 
result  of  AISI's  work  to  provide  more  time  and  support  for  professional  development, 
and  to  increase  dedication  to  collaborative  decision-making  involving  a  wider  range  of 
participants.  If  there  is  any  single  area  in  which  AISI  is  most  advancing  policy  changes 
at  the  provincial  level  and  throughout  the  wider  culture  of  education,  it  is  in  this  crucial 
domain  of  collective  learning,  connectivity  among  schools,  and  overall  enhancement  of 
capacity. 

Along  with  changes  in  teaching  have  come  shifts  in  how  leadership  is  developed  in 
schools.  Leadership  is  no  longer  confined  to  the  principal's  or  superintendent's  office 
but  is  increasingly  being  spread  throughout  the  professional  community,  where  it  retains 
a  close  connection  to  classroom  learning.  This  is  a  significant,  inspiring  and  world- 
leading  aspect  of  the  changing  culture  of  education  in  Alberta,  at  time  when  teacher 
leadership  is  little  more  than  a  cliche  in  most  other  jurisdictions. 

There  remain  three  ways  in  which  AISI  does  not  yet  seem  to  have  influenced  the  wider 
culture  of  education  and  educational  change  in  Alberta.  The  first  concerns  the  existence 
of  prior  and  parallel  cultures  of  hierarchical  leadership  and  administration  in  a  number 
of  districts.  The  second  is  related  to  the  central  administration  of  Alberta  Education  and 
its  impact  on  school  and  district  cultures.  The  final  limitation  relates  to  the  need  for 
extension  of  networking  activities  across  districts  to  promote  optimal  learning  among 
educators. 

AISI  initiatives  and  the  ways  in  which  they  are  developed  are  often  absorbed  into  the 
existing  cultures  of  administration  within  school  districts,  which  they,  in  turn,  seem  to 
amplify.  Districts  organized  on  hierarchical  lines  with  a  narrative  of  management  tend 
to  decide  on  and  impose  a  focus,  invest  in  external  packages  and  trainers,  use  resources 
to  put  coordinators  into  the  district  office  thereby  swelling  the  ranks  of  administration, 
and  create  little  independent  connectivity  among  schools.  Lines  of  control  are  top- 
down,  implementing  administratively  selected  initiatives  and  making  it  difficult  for 
schools  to  learn  from  each  other.  Districts  with  an  ethic  or  narrative  of  service  provide 
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more  trust,  invest  more  heavily  in  relationships,  and  secure  commitment  to  common 
goals,  but  patterns  of  implementation  are  still  paternalistic,  and  staff  overload  is  heavy. 
This  restricts  the  opportunities  for  organizational  learning. 

Some  of  the  educators  interviewed  in  the  15  districts  studied  in  the  two  qualitative 
components  of  the  multiple  perspectives  review  communicated  that  Alberta  Education 
is  not  perceived  as  being  part  of  a  wider  learning  community.  So  far,  AISFs  flexible, 
adaptable,  participatory  and  networked  approach  with  its  broad  conception  of  learning 
has  interrupted  this  perception,  but  more  as  a  refreshing  alternative  to  larger 
transactional  approaches  that  leave  educators  wary  of  other  government  initiatives. 

The  transactional  model  of  Alberta  Education  is  not  unusual  and  seems  to  operate  like 
most  other  education  ministries.  It  is  perceived  by  AISI  participants  as  a  system  of 
central  policy  development  that  is  then  implemented  through  the  hierarchical  authority 
of  individual  superintendents  and  line-managed  by  principals  below  them.  This  system 
is  often  well  organized  to  implement  common  programs  and  strategies.  It  is  less  suited 
to  innovation  and  to  developing  practices  that  require  local  discretion.  On  the  ground, 
AISI  is  in  tension  with  the  existing  policy  culture.  But  as  AISI  progresses  further  and 
policy  goals  also  begin  to  incorporate  more  innovative  elements  suited  to  knowledge 
economy  goals,  this  tension  could  become  a  creative  and  energizing  one  of  productive 
disturbance. 

Within  schools,  AISI  appears  to  be  eradicating  the  longstanding  presence  of  privacy  in 
the  culture  of  teaching.  PLCs  among  teachers  and  administrators  have  been  established 
to  study  the  real  and  most  daunting  problems  as  well  as  the  most  inspiring  and 
innovative  challenges  facing  schools  and  then  to  develop  new  strategies  for  responding 
to  them.  These  are  not  just  individual  teacher  opportunities  but  collective  professional 
responsibilities.  This  is  an  enormous  achievement  that  has  eluded  educational 
reformers  in  many  other  jurisdictions  around  the  world. 

The  greater  challenge  of  privacy  and  isolation  that  classroom  teachers  have  experienced 
in  the  past  is  now  a  different  one.  It  is  the  privacy  and  isolation  that  insulates  and 
separates  school  districts.  This  inhibits  the  potential  for  learning  across  schools  and 
projects  independently  of  detailed  district  control.  While  some  districts  have  been  able 
to  surmount  these  problems,  systemic  decisions  now  have  to  be  made  on  behalf  of 
teachers  and  learners  in  the  others: 

Is  AISI  essentially  going  to  an  outlier  to  or  even  a  safety  valve  for  a  relatively 
traditional  provincial  system  of  education? 

Or,  in  a  context  of  the  province's  reinvention  as  a  competitive  and  innovative 
knowledge  economy  within  an  increasingly  diverse  community,  can  AISI  now  be  the 
catalyst  for  a  more  participatory  and  decentralized  process  of  policy  development? 

In  other  words,  can  AISI  create  a  renewed  and  reinvented  relationship  between  the 
central  ministry  and  its  districts  as  well  as  among  the  districts  themselves? 
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By  challenging  districts  to  innovate,  demanding  accountability,  and  infusing  a  level  of 
uncertainty  around  the  maintenance  of  funding,  AISI  already  provides  a  source  of 
productive  disruption  of  business-as-usual  in  districts.  It  capitalizes  on  local  ingenuity 
and  inventiveness  and  empowers  educators  to  explore  new  strategies  for  engaging 
reluctant  learners.  It  is  unlikely  that  this  innovation  and  creativity  would  occur  without 
AISI.  Increased  support  for  school  networking  across  districts  will  spread  and  accelerate 
these  processes. 

Recommendations 

AISI's  change  architecture,  theory  of  action,  and  sensitivity  to  issues  impacting  Alberta's 
diverse  schools  are  unusually  sophisticated  and  responsive.  Yet  even  the  best  change 
initiatives  can  be  strengthened.  This  brings  us  to  our  recommendations: 

1.  develop  improved  ways  of  collecting  and  compiling  provincial  achievement  data 
that  will  make  it  possible  to  trace  the  impact  of  complex  but  distinct  initiatives 
like  AISI; 

2.  create  leadership  and  support  systems  for  teachers  and  administrators  involved 
in  AISI  projects  to  access  existing  data  bases,  request  and  receive  data  analysis 
services,  and  design  their  own  instruments  and  indicators  of  accountability  that 
are  appropriate  to  their  project  goals; 

3.  extend  AISI  project  content  and  processes  towards  greater  involvement  of 
parents,  community  members,  businesses,  universities  and  other  partners; 

4.  increase  AISI's  attention  to  and  impact  regarding  innovation  and  improvement 
in  high  schools,  with  particular  reference  to  increasing  Alberta's  relatively  low 
rates  of  high  school  completion; 

5.  invest  in  province- wide  networks  that  cut  across  districts,  that  reach  beyond 
annual  conferences  and  that  incorporate  proven  design  principles  of  effective 
network  architectures  that  have  clear,  positive  impacts  on  system-wide  outcomes 
for  students; 

6.  develop  leadership  skill  and  capacity  among  all  principals  and  district-level 
leaders  so  that  the  effectiveness  of  AISI  projects  does  not  suffer  when  existing 
leadership  capacity  in  particular  schools  and  districts  is  not  strong; 

7.  embed  AISI  into  Alberta  Education  as  an  integrated  policy  strategy.  Do  this 
without  diminishing  the  attention,  resources,  advocacy  and  professional 
development  regarding  the  distinctive  approaches  to  professionally  driven, 
locally  adaptable  and  laterally  networked  processes  of  improvement  and 
innovation  that  AISI  has  championed. 
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These  seven  recommendations  are  clustered  into  six  broad  and  interconnected  thematic 
areas  that  warrant  attention  for  the  further  success  of  AISI  in  coming  years: 


1.  Preservation 

2.  Purpose  and  focus 

3.  Impact 

4.  Culture 

5.  Structure  and  funding 

6.  Leadership 


1.  Preservation.  The  research  team  finds  much  that  is  of  value  in  AISI.  Educators 
consistently  state  that  AISI  is  making  tremendous  contributions  to  the  advancement  of 
the  teaching  profession  by  giving  educators  new  sets  of  skills  for  understanding  student 
learning  and  assessment  practices.  Within  districts,  educators  appreciate  the 
opportunity  to  learn  from  their  colleagues  in  other  schools,  and  venues  such  as  annual 
AISI  conferences  enable  educators  to  meet  and  exchange  their  learning  with  others  from 
throughout  the  province.  This  culture  of  inquiry  and  exchange  then  emboldens 
educators  to  explore  new  ideas  that  they  can  adapt  to  local  circumstances  to  better  serve 
their  students  and  their  communities.  Whatever  changes  might  be  made  to  AISI  in  the 
future,  these  should  ensure  the  preservation  of  these  praiseworthy  principles  and  their 
accompanying  strategies. 

2.  Purpose  and  focus.  AISI's  positive  benefits  are  significant.  They  can  be  used  to 
contribute  to  the  honing  of  AISI's  purpose  and  focus.  At  the  start  of  its  second  decade, 
AISI  acknowledges  that  it  no  longer  is  an  initiative  but  a  strategy  that  has  proven  its 
value  and  become  an  anchor  of  the  province's  school  improvement  efforts.  A  more 
systemic  focus  on  areas  that  proved  difficult  for  AISI  in  its  first  ten  years — such  as  high 
school  improvement  or  increased  parent  and  community  engagement  in  schools — is 
now  warranted.  More  exphcit  emphasis  on  innovation  and  sustainability,  with 
particular  reference  to  21^'  century  learning,  may  also  be  of  value.  These  latter  themes 
are  already  present  in  AISI  Cycle  4's  recommendations.  They  merit  even  greater 
emphasis  in  the  second  decade  and  overall  future  of  AISI. 

3.  Impact.  AISI  is  a  school  improvement  strategy,  not  an  experimental  or  quasi- 
experimental  research  design.  Nonetheless,  accountability  and  improvement  arguments 
make  it  prudent  to  develop  systemic  support  that  could  enable  in-school  inquiry  and 
systemic  evaluation  to  make  more  efficient  and  effective  use  of  data  to  better  measure 
AISI's  impact.  The  research  team  therefore  recommends  that  AISI  establish  an  AISI 
Institute  of  Data  (ADD)  to  collect,  compile,  and  compute  data  related  to  individual 
students,  schools,  and  districts  for  use  by  policy  makers  and  AISI  projects  alike.  The 
team  also  recommends  that  AID  also  provides  customized  support  for  schools  and 
projects  to  select  and  align  impact  data  in  relation  to  project  goals. 
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4.  Culture.  AISFs  culture  is  dynamic,  intellectually  rigorous  and  sustainable  Educators 
were  consistently  appreciative  of  a  policy  of  innovation  that  genuinely  enhanced 
teaching  and  learning  in  a  manner  that  respected  their  insights  and  promoted  their 
continual  professional  growth.  At  the  same  time,  AISFs  culture  could  be  enriched 
further  if  AISI  is  even  more  proactive  about  building  lateral  learning  networks  across  as 
well  as  within  districts.  AISI  educators  have  established  a  positive  culture  of  sharing 
practices  and  celebrating  successes  but  they  might  also  benefit  from  a  more  challenging 
culture  of  frank  acknowledgment  of  stalled  innovations,  flat  achievement  results  or 
differences  among  schools  in  making  progress.  One  of  the  challenges  for  AISI  is  not 
only  to  amplify  district  cultures  but  also  to  challenge  and  change  them  where 
appropriate. 

5.  Structure  and  funding.  AISI  can  be  enhanced  by  rethinking  its  structure  and  funding 
strategy.  A  number  of  schools  and  districts  would  welcome  a  more  flexible  funding  and 
proposal  cycle,  with  the  possibility  of  proposals  that  extend  beyond  the  usual  three-year 
limit.  This  could  enable  districts  to  go  deeper  into  especially  challenging  areas  without 
the  uncertainty  and  need  for  premature  closure  that  is  sometimes  built  into  three-year 
ceilings  on  grant-funded  projects.  Districts  and  projects  can  be  supported  through 
targeted  funding  to  network  with  one  another  in  more  sustained  ways.  If  funded,  the 
AISI  Institute  for  Data  (AID)  could  help  to  gauge  the  benefits  of  longer  and  more 
flexible  projects  versus  those  that  are  more  limited  in  scope  and  duration. 

6.  Leadership.  Like  many  change  strategies,  AISI  generally  has  assigned  leadership  a 
secondary  status  in  its  theory  of  action.  AISI  tends  to  amplify  existing  leadership 
cultures  without  an  explicit  acknowledgment  that  they  may  be  more  or  less  capable  of 
supporting  AISI  activities.  For  instance,  teacher  leadership  in  AISI  has  been  encouraged 
as  a  fundamental  principle  and  favored  strategy  but  without  concomitant  attention  to  its 
implications  for  leadership  by  principals,  superintendents,  and  other  district  staff.  Yet 
the  research  literature  indicates  that  for  teacher  leadership  to  be  developed  in  a  long- 
term,  sustainable  way,  ongoing  technical  support  is  crucial  for  principals, 
superintendents,  and  district-level  staff. 

Several  school  districts  have  used  AISI  funds  to  develop  PLCs  not  only  for  teachers  but 
also  for  administrators.  These  are  helping  administrators  to  identify  areas  in  which  they 
can  help  teachers  to  grow  professionally  and  especially  as  instructional  leaders.  This 
work  appears  to  be  especially  valuable  in  improving  the  quality  of  instruction  at  the 
high  school  level.  Enriching  such  administrative  PLCs  by  extending  them  in  networks 
across  district  lines  and  enabling  them  to  benefit  from  the  challenges  as  well  as  support 
of  colleagues  in  other  districts  is  a  natural  extension  of  the  advantages  of  networks  for 
teachers'  ongoing  professional  development.  Effective  networking  does  not  come 
naturally  though,  and  developing  skills  of  network  leadership  should  be  a  priority  in  the 
future. 
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Conclusion 

AISI  is  an  impressive  change  strategy  that  is  perhaps  without  parallel  in  the  world 
today.  It  contributes  to  teacher  development  and  educational  change  in  a  manner  that  is 
stable,  steady,  and  credible  among  the  educators  it  most  seeks  to  impact.  AISI 
leadership  is  transparent,  responsive,  and  trustworthy. 

AISI  has  built  a  solid  foundation  to  further  evolve  and  address  some  of  the  most 
tenacious  problems  in  educational  change  today.  In  the  years  ahead,  AISI  leaders  should 
build  upon  their  many  accomplishments  and  expand  the  most  important  themes  and 
strategies  of  AISI  into  new  arenas.  AISI  should  further  promote  learning  across  district 
lines  and  should  increase  parents  and  community  engagement  in  schools.  More 
concerted  efforts  and  sustained  support  need  to  be  provided  to  high  schools  to  engage 
students  and  to  transform  learning.  In  general,  more  flexible  and  also  more  targeted 
approaches  to  funding  and  funding  cycles  may  help  achieve  these  goals. 

AISI  is  already  promoting  some  of  these  changes  in  its  new  cycle  of  projects.  It  is 
imperative  that  AISI  act  decisively  and  boldly  in  leading  the  changes.  AISI  has  a 
unique  change  architecture.  It  treats  the  learning  of  students,  teachers  and  organizations 
not  as  a  line,  or  even  a  circle,  but  as  a  complex,  interlocking  mosaic.  AISI  is  a  complex 
model  of  improvement  and  innovation  and  also  a  transparent  and  participatory  one. 
This  is  why  it  enjoys  increasing  visibility  not  only  in  Alberta  or  Canada  but  also  among 
policy  makers  in  other  nations.  AISFs  continued  progress  within  one  of  the  world's 
very  highest  performing  systems  will  be  keenly  observed  by  scholars  of  educational 
change  and  policy  makers  focused  on  improving  student  learning  from  around  the 
globe. 
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Chapter  1:  Investigating  AISI  

By  Andy  Hargreaves 

1.  Introduction 

On  April  29,  2004,  former  Prime  Minister  Paul  Martin  gave  an  address  in  Washington 
in  which  he  favorably  compared  the  metaphor  of  Canada's  bilingual,  multicultural 
mosaic,  to  that  of  the  common  melting  pot  or  stew  into  which  all  differences  are 
dissolved  in  the  United  States.  Despite  the  criticism  that  hidden  hierarchies  persist 
within  this  multicultural  mosaic  and  that  indeed  the  very  first  use  of  mosaic  imagery  in 
a  social  sense  was  as  a  critique  of  the  existence  of  a  vertical  mosaic  in  Canada  (Porter, 
1965),  the  metaphor  remains  appropriately  central  and  essential  to  Canadian  identity. 
More  than  this,  the  mosaic  metaphor  first  emerged  in  Canada's  prairie  heartland  where 
it  referred  to  a  particular  style  of  architecture. 

Learning  and  learning  communities  are  also  well  captured  by  the  metaphor  of  the 
mosaic,  as  one  author's  namesakes  already  recognized  more  than  a  decade  ago  (D 
Hargreaves,  1994).  Learning  and  communities  of  learning  are  made  up  of  diverse  pieces 
-  artistic  and  scientific,  improvised  and  memorized,  acquired  individually  or 
cooperatively  with  others  -  that,  with  suitable  architectural  vision,  guidance  and  design, 
make  up  a  complex  but  clear  unity  of  achievement. 

The  architecture  of  the  decade-old  Alberta  Initiative  for  School  Improvement  (AISI) 
and  indeed  of  any  highly  effective  educational  system  such  as  Alberta's  or  Finland's 
(the  only  jurisdiction  that  outperforms  Alberta  on  OECD's  international  PISA  tests  of 
educational  achievement),  in  a  fast-moving,  knowledge-driven  world  of  innovation  and 
creativity,  is  also  best  thought  of  as,  at  its  best,  a  complex  but  coherent  mosaic. 
Following  futurist  Alvin  Toffler  (1990),  it  might  even  be  regarded  as  a  moving  mosaic  - 
a  dynamic  kaleidoscope  of  local,  school  and  district-determined  improvement  and 
innovation,  that  has  a  shifting  but  clearly  definable  pattern  embodying  the  creativity, 
flexibility  and  adaptability  that  Toffler  advocated. 

Sophisticated  learning  of  the  kind  that  encompasses  21"*'  century  skills  and  that  is 
increasingly  regarded  as  necessary  for  advanced  knowledge  economies  (OECD,  2008) 
is  not  a  circle  or  a  line.  It  runs  neither  in  predictable  steps  nor  in  orderly  cycles.  It  is  a 
complex  system  and  process  -  much  more  like  a  moving  mosaic.  This  is  the  essence 
and  the  aspiration  of  AISI  at  its  best.  Unlike  many  other  less  effective  systems,  AISI 
does  not  operate  as  a  set  of  controlled  interventions,  orchestrated  in  a  linear  or  layered 
way  through  levels  of  implementation  from  the  top.  At  the  same  time,  as  a  more 
complex  system,  it  always  has  to  be  watchful  of  turning  into  a  rubble  of  unrelated 
fragments  in  which  no  clear  or  coherent  vision  of  learning  can  be  discerned  at  all. 
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AISI  stands  between  the  dynamics  of  free  will  and  determinism  -  or  between  what 
complexity  theorists  call  emergence  and  design  (Capra,  2002).  As  a  complex  collection 
of  local  pieces  of  improvement  and  instruction,  brought  together  to  try  and  form  a 
coherent  whole,  the  success  or  failure  of  AISI,  as  of  all  deliberately  constructed 
complex  systems,  resides  in  its  particular  principles  of  architectural  design  and  their 
resulting  effects.  Too  much  design  and  AISI  becomes  just  another  mechanism  of  top- 
down  implementation.  Too  much  emergence,  and  innovation  as  well  as  improvement 
efforts  can  be  inconsistent  in  quality,  and  lack  any  kind  of  collective  or  system-wide 
coherence.  In  complex  improvement  designs  like  AISI,  architecture  is  everything. 

So  what  is  the  architectural  design  of  AISI?  How  clear  and  appropriate  is  its  vision? 
What  has  been  its  impact  and  effects,  and  how  easily  are  these  disentangled  from 
Alberta's  other  educational  initiatives?  Are  the  efforts  and  impacts  of  AISI  sustainable? 
And  could  they  be  achieved  more  easily  or  prudently  by  other  means? 

To  address  these  questions,  a  multiple  perspectives  review  of  AISI  was  commissioned 
by  Alberta  Education  in  2008.  The  review  was  conducted  by  individuals  and  teams 
located  in  British  Columbia,  Newfoundland,  Boston  in  the  United  States,  and  Finland.  It 
encompassed  methods  of  investigation  and  inquiry  that  included  quantitative  analysis  of 
existing  data  sets,  qualitative  case  studies  of  schools  and  districts,  and  comparative 
analysis  in  relation  to  other  reform  and  improvement  strategies  elsewhere  and  over 
time.  The  review  was  conducted  between  May- August  2009. 

This  chapter  of  this  multiple  perspectives  review  describes  the  nature  of  AISI  and  the 
background  to  the  conduct  of  this  review.  It  states  the  questions  addressed  by  the  review 
and  introduces  the  sub-components  of  the  review  that  follow  in  the  ensuing  chapters. 
Following  chapters  on  each  of  the  sub-components  of  the  review,  the  report  closes  by 
presenting  the  overall  findings  along  with  a  set  of  recommendations  for  the  future  of  AISI. 

2.  A  Brief  Description  of  AISI 

The  nature  of  and  background  to  AISI  are  described  by  Alberta  Education  in  the 
following  terms: 

The  Alberta  Initiative  for  School  Improvement  (AISI)  is  a  bold  approach  to 
improving  student  learning  by  encouraging  teachers,  parents,  and  the  community 
to  work  collaboratively  to  introduce  innovative  projects  that  address  local  needs. 
Initiated  in  1999  by  the  Alberta  Government  and  its  partners,  AISI  provides 
targeted  funding  to  school  authorities  to  improve  student  learning  and  enhance 
student  engagement  and  performance.  More  than  $500  million'  has  been  invested 
in  this  initiative  to  continuously  improve  student  learning  in  Alberta.  After  three 
successful  three-year  cycles  of  the  Alberta  Initiative  for  School  Improvement 
(2000-2009),  AISI  is  now  in  its  fourth  cycle,  2009-2012  (AISI  Education  Partners, 
2008,  p.  i). 


At  the  time  of  printing  this  report  the  amount  is  now  $625  million. 
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AISI  was  first  initiated  a  decade  ago  as  a  result  of  the  combined  efforts  and 
commitments  of  Alberta  Education  and  its  partners:  the  Alberta  Teachers'  Association 
(ATA),  Alberta  School  Boards  Association  (ASBA),  Alberta  School  Councils' 
Association  (ASCA),  Association  of  School  Business  Officials  of  Alberta  (ASBOA), 
and  the  College  of  Alberta  School  Superintendents  (CASS);  in  2000,  the  University 
Faculties  of  Education  joined  the  partnership. 

In  AISI,  participating  school  authorities  (public,  separate.  Francophone,  charter  and 
private),  propose  and  then,  if  criteria  set  by  the  AISI  partners  are  met,  receive  resources 
for  self-designed  projects  that  focus  on  improving  student  learning,  engagement  and 
performance  in  ways  that  suit  local  circumstances.  Projects  have  a  wide  array  of 
emphases,  though  in  each  particular  cycle  there  is  a  tendency  for  a  substantial 
proportion  (up  to  40%)  to  cluster  around  one  particular  theme  like  differentiated 
instruction,  professional  learning  communities,  or  assessment  for  learning. 

AISI  has  been  organized  in  three-year  cycles.  At  the  same  time,  there  is  increasing 
encouragement  for  projects  to  build  on  prior  ones  where  appropriate.  Projects  may  be 
one,  two,  or  three  years  in  duration  and  there  is  an  expectation  that  what  has  been 
learned  will  be  integrated  and  built  upon.  Each  cycle  tends  to  have  a  particular 
emphasis  or  "feel"  with  Cycle  1  being  characterized  by  a  great  diversity  of  local 
projects,  Cycles  2  and  3  being  defined  by  greater  efforts  to  create  coherence  among 
projects,  especially  at  the  district  level,  and  the  new  Cycle  4  placing  particular  stress  on 
engaging  students,  building  leadership  capacity  within  districts,  and  networking  of 
schools  and  projects  across  them. 

Administration  of  AISI  within  districts  and  other  school  authorities,  and  ways  of 
achieving  cohesion  among  projects  within  districts  do  not  follow  one  pattern,  as  Sumara 
and  Davis'  chapter  illustrates  in  its  comparison  of  three  different  districts.  Across  the 
province,  projects  are  connected  in  annual  conferences  and  in  other  occasional 
province-wide  meetings  and  events,  as  well  as  by  activity  on  the  AISI  website. 

AISI  is  funded  at  around  $70-75  million  per  annum  and  involves  more  than  95%  of  the 
province's  schools  in  self-designed  and  initiated  innovation  and  improvement  projects. 
This  comprises  about  2%  of  the  province's  operating  education  budget.  About  1,700 
projects  have  been  funded  so  far. 

AISI  places  great  emphasis  on  professional  learning  and  inquiry  as  a  central  element  of 
improvement,  and  on  thorough  procedures  of  accountability  that  include  narrative 
accounts  of  intended  goals,  plans,  activities  undertaken,  lessons  learned  and  indicators 
of  impact.  Impact  indicators  commonly  include  readily  available  provincial  instruments 
such  as  provincial  achievement  tests  (PATs),  as  well  as  survey  instruments  of 
satisfaction  levels  etc.  They  can  and  sometimes  also  do  entail  self-designed  instruments 
and  standardized  assessments. 

The  activities  and  impacts  of  AISI  have  already  been  extensively  documented.  All 
cycles  are  subject  to  careful  and  rigorous  evaluation  (e.g.,  AISI,  2004,  2008).  Provincial 
Reports  such  as  these  evaluations  also  include  detailed  discussions  of  AISI's  origins, 
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evolution,  implementation  strategies  and  assessment.  Recent  reports  by  Alberta 
Education  (2009a,  2009b)  have  also  disseminated  AISI  project  themes  on  a  range  of 
topics.  In  addition,  aspects  of  AISI  have  been  studied  by  university  researchers. 
Subjects  that  have  been  investigated  include  high  school  completion  (Gunn,  Chorney,  & 
Poulsen,  2008),  leadership  and  sustainabihty  (Foster,  Wright,  &  McRae,  2008),  the  role 
of  parents  and  community  in  supporting  student  success  (Steinmann,  Malcolm,  Connell, 
Davis  &  McMann,  2009),  and  First  Nations,  Metis,  and  Inuit  learners  (Gunn,  Pomahac, 
Striker,  &  Tailfeathers,  2009). 

In  international  terms,  AISI  is  an  extremely  unusual  system- wide  change  strategy  that  is 
attracting  increasing  attention  worldwide  because  of  its  distinctive  differences  from 
other  reform  strategies  in  what  is  an  exceptionally  high  performing  educational  system. 
Its  commitment  to  school-based  and  district-based  initiatives  with  targeted  funding 
includes  almost  all  the  schools  in  the  province  in  a  concerted  effort  at  systemic  change. 
It  encourages  local  initiative  and  expresses  high  degrees  of  professional  trust  within 
what  are  nonetheless  some  of  the  most  stringent  systems  of  external  accountability  and 
achievement  testing  in  the  nation  and  the  world.  AISI  is  an  enigma  of  change.  The 
purpose  of  this  report  is  to  investigate  this  enigma  -  to  make  its  architecture  and  its 
impact  explicit,  with  a  view  to  making  judgments  about  its  success  so  far  and 
recommendations  regarding  its  future. 


3.  The  Multiple  Perspectives  Review 

In  October  2008,  as  AISI  prepared  for  its  fourth  Cycle  and  tenth  year  of  operation,  the 
Alberta  Initiative  for  School  Improvement  held  a  province-wide  Colloquium  to  take 
stock  of  the  progress  of  AISI  to  date,  and  to  help  set  directions  for  the  future.  Involving 
key  stakeholders,  AISI  Education  Partners  and  staff  from  the  School  Improvement 
Branch  of  Alberta  Education  in  which  AISI  is  administered,  and  school  jurisdiction 
presenters  of  a  number  of  AISI  projects,  the  Colloquium  established  a  characteristically 
open  and  transparent  process  of  dialogue  and  reflection  about  AISI's  strengths  and 
limitations,  and  about  the  small  and  large  adjustments  that  may  be  needed  in  reshaping 
its  future. 

AISI  invited  to  the  Colloquium  several  researchers  who  acted  as  critical  friends  for  the 
initiative.  They  participated  in  dialogue,  observed  AISI  presentations,  interacted  with 
stakeholders,  and  responded  to  a  range  of  the  extensive  documentation  on  AISI  and 
evaluations  of  AISI  that  had  been  produced  to  date.  Robert  Crocker,  formerly  of 
Memorial  University  in  Newfoundland,  contributed  his  considerable  expertise  in 
experimental  and  survey  design  as  well  as  statistical  meta-analysis  of  existing  data  sets 
to  raise  issues  regarding  the  measurement  of  AISI's  impact.  Dennis  Sumara  and  Brent 
Davis,  then  at  the  University  of  British  Columbia,  presented  their  field-leading  work  on 
complexity  theory  and  its  uses  in  education,  and  provided  initial  feedback  on  how  AISI 
may  or  may  not  be  operating  as  a  complex  system.  Andy  Hargreaves,  formerly  of  the 
Ontario  Institute  for  Studies  in  Education  and  now  at  Boston  College,  offered  some 
initial  observations  on  AISI's  architecture  as  a  change  strategy  and  its  similarity  to  and 
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difference  from  other  systemic  change  strategies.  Finally,  Pasi  Sahlberg,  Director  of  the 
Centre  for  International  Cooperation  and  Mobility  in  Finland,  provided  his  observations 
from  an  international  policy  perspective. 

Following  the  Colloquium,  these  contributors  were  invited  to  undertake  deeper  research 
with  a  slightly  widened  team  that  would  comprise  a  multiple  perspectives  review  on  the 
design,  impact  and  future  of  AISI,  including  its  sustainability.  Sixteen  research 
questions  were  finalized  in  February  and  March  2009  through  a  process  of  consensus  by 
the  research  team  and  Alberta  Education.  Five  of  these  were  overarching  questions: 

1.  What  is  the  distinctive  theory-in-action  (change  architecture)  of  AISI? 

2.  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI?) 

3.  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

4.  Would  the  values  of  AISI  continue  without  funding? 

5.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

These  were  supplemented  by  eleven  subsidiary  questions: 

1.  What  have  been  the  successes  of  AISI,  as  assessed  from  multiple  perspectives? 

2.  What  are  both  the  obvious  and  the  subtle  impacts  of  AISI? 

3.  How  is  AISI  lived  and  practiced  by  educators? 

4.  What  are  the  change  processes  at  play  for  administrators  and  teachers? 

5.  What  have  been  the  difficulties  and  challenges  of  AISI? 

6.  Has  AISI  encouraged  school  authorities  to  try  new  things? 

7.  Has  AISI  encouraged  those  involved  to  take  risks  and  to  be  more  innovative? 

8.  What  are  the  opportunities  to  expand  the  measures  of  AISI  projects  from  the 
vantage  point  of  complexity  theory? 

9.  What  are  the  opportunities  to  disseminate  knowledge  generated  by  AISI  by 
using  its  networks  and  complexity  thinking?  Have  these  opportunities  been 
used  to  promote  change  across  AISI  jurisdictions? 

10.  How  has  AISI  influenced  policy  developments  at  the  school,  jurisdictional,  and 
provincial  levels? 

11.  What  are  the  implications  of  the  research  findings  for  AISI  as  a  work  in 
progress? 

Andy  Hargreaves  served  as  overall  project  coordinator  for  three  research  teams  and  a 
critical  friend  who  was  also  appointed  to  the  project. 

The  first  research  component  was  conducted  by  Robert  Crocker.  This  took  the  form  of  a 
meta-analysis  of  existing  provincial  data  sets  concerning  tested  achievement  results,  as 
well  as  survey  data  of  satisfaction  levels  for  parents,  students  and  teachers,  in  relation  to 
the  effects  of  AISI.  This  study  conducted  further  analysis  of  data  from  AISI  Cycle  2  and 
preliminary  analysis  of  Cycle  3  data  to  try  and  ascertain  whether  there  were  observable 
changes  over  time  and  across  main  AISI  themes  and  strategies,  the  extent  to  which 
these  changes  could  be  attributed  to  AISI,  and  whether  some  AISI  themes  and/or 
strategies  were  more  effective  than  others  in  relation  to  valued  outcomes.  In  addition, 
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this  study  and  the  chapter  arising  from  it,  evaluates  the  suitabiUty  of  existing  data  sets 
and  data  gathering  processes  for  determining  the  impact  of  AISI  and  makes  suggestions 
for  future  improvement  in  subsequent  cycles.  This  component  of  the  project  is  reported 
in  the  next  chapter. 

Chapter  3  reports  the  results  of  a  second  component  of  this  multiple  perspectives  review 
that  responds  to  four  questions  originally  posed  in  October  2008  by  Alberta's  Deputy 
Minister  of  Education,  Keray  Henke: 

1 .  What  is  the  value  of  the  AISI? 

2.  Why  couldn't  jurisdictions  do  this  anyway? 

3.  Would  the  values  of  AISI  continue  without  funding? 

4.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how? 

Dennis  Sumara  and  Brent  Davis  bring  an  understanding  of  complexity  theory  to  these 
questions  to  study  how  AISI  has  affected  educational  cultures  within  different  school 
districts  and  whether  any  identified  benefits  might  be  sustainable.  Their  analysis  and  the 
insights  from  it  are  sharpened  by  a  qualitative  interpretation  of  AISI  design  and 
development  in  relation  to  the  prior  patterns  of  culture  and  leadership  that  existed  and 
persist  in  three  strikingly  contrasting  districts. 

In  Chapter  4,  Dennis  Shirley  and  Lori  McEwen  present  their  qualitative  condensed  case 
study  of  12  varied  and  geographically  dispersed  school  districts.  This  study  employed 
interviews  and  focus  group  discussions  as  well  as  analysis  of  school,  district  and  project 
documents,  to  gather  data  about  the  meaning  and  value  of  AISI  among  educators  and 
district  personnel  involved  in  and  responsible  for  design,  implementation,  and 
assessment  of  AISI  activities  at  the  district  level.  This  up-close  view  of  AISI  on  the 
ground  provides  evidence  of  the  perceived  architecture,  impact,  strengths  and 
challenges  of  AISI  as  well  as  the  context  in  which  AISI  operates  among  those  who  are 
most  closely  engaged  with  it. 

Chapters  5  and  6  draw  on  the  findings  and  evidence  of  the  three  preceding  chapters  and 
compare  the  architecture  and  impact  of  AISI  to  other  reform  strategies  over  time  and  in 
other  places.  Andy  Hargreaves  compares  AISI  to  four  ways  of  change  that  have 
characterized  educational  change  over  the  last  half-century  and  reflects  on  these  to 
make  judgments  about  the  strengths,  limitations  and  future  possibilities  for  AISI.  Pasi 
Sahlberg  refers  to  his  extensive  knowledge  of  other  reform  examples  in  Finland  and 
across  the  world  to  determine  the  distinctiveness  of  AISI  and  also  delineate  its  best 
possible  ways  forward. 

Finally,  Chapter  7  and  Chapter  8  draw  together  all  the  key  findings  across  the  sectors  as 
well  as  some  recommendations  arising  from  them.  These  are  not  just  a  collation  of  the 
results  of  the  separate  sub-components  of  the  review,  but  the  product  of  intense 
discussion  and  analysis  in  a  three-day  retreat  for  all  the  teams  in  August  2009,  after  the 
first  drafts  of  their  reports  had  been  prepared. 
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4.  Conclusion 

This  review  is  especially  timely.  It  has  been  commissioned  ten  years  into  one  of  the 
most  remarkable  and  enigmatic  approaches  to  school  improvement  and  innovation  on  a 
system-wide  basis  in  the  world.  AISI  involves  a  substantial  sum  of  provincial 
expenditure,  and  especially  at  a  time  of  economic  downturn  it  is  important  to  review 
what  AISI  is  and  might  be,  what  it  has  achieved  and  might  achieve  in  the  future,  and 
what  elements  of  orientation  and  design  may  need  to  be  reconsidered  and  reconstructed. 

Beyond  Alberta,  as  new  questions  are  being  raised  about  what  young  people  need  to  be 
able  to  learn  in  a  context  of  great  economic  and  social  reconstruction,  AISI  is  attracting 
considerable  global  attention  as  a  distinctive,  even  unique  approach  to  educational 
change.  An  assessment  of  how  it  works  and  what  it  is  worth  is  therefore  especially 
opportune  at  this  moment.  Alberta  is  a  world  leader  in  educational  standards.  AISI  is 
internationally  on  the  leading  edge  of  approaches  to  innovation  and  improvement.  This 
is  a  review  from  which  many  might  benefit,  inside  and  outside  the  province.  The 
learning  mosaic  is  now  a  global  mosaic.  Having  designed,  conducted,  coordinated  and 
presented  this  review  in  approximately  six  months,  the  multiple  perspectives  team 
hopes  that  its  efforts  and  insights  will  prove  helpful  to  the  province  and  to  all  who  care 
about  developing  better  improvement  strategies  for  the  learning  of  our  young  people  in 
the  21^^  century. 
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Chapter  2:  Rethinking  the  AISI  Research  Model:  Secondary  Data 
Analysis  and  Future  AppHcations  

By  Robert  Crocker 

1.  Introduction 

1.1  Purpose 

This  study  had  two  main  purposes.  The  first  was  to  conduct  further  analysis  of  data 
from  AISI  Cycle  2  and  preliminary  analysis  of  Cycle  3  data.  More  specifically,  the  goal 
was  to  address  the  following  three  questions: 

1.  Are  there  observable  changes  over  time  and  across  main  AISI  themes  and 
strategies? 

2.  To  what  extent  can  these  changes  be  attributed  to  AISI  rather  than  to  extraneous 
factors? 

3.  Are  some  AISI  themes  and/or  strategies  more  effective  than  others  in  bringing 
about  gains  in  valued  outcomes? 

The  second  purpose  was  to  review  the  quantitative  analysis  model  and  data  gathering 
processes  of  the  Alberta  Initiative  for  School  Improvement  (AISI)  and  make 
suggestions  for  enhancing  the  model  and  the  quality  of  data  in  subsequent  cycles.  More 
specific  questions  under  this  purpose  are: 

1.  Do  alternative  approaches  to  analysis  exist  that  might  shed  further  light  on  AISI 
project  impacts? 

2.  Can  these  alternatives  be  effectively  applied  to  the  existing  data  base, 
specifically  for  Cycle  2  and  the  first  two  years  of  Cycle  3,  in  ways  that  would 
add  value  beyond  the  existing  reports  and  improve  the  analysis  in  subsequent 
cycles? 

3.  Can  the  AISI  research  model  be  modified  for  Cycle  4  in  ways  that  can  reduce 
some  of  the  limitations  of  existing  research  designs? 

4.  Can  existing  data  bases  be  used  to  investigate  provincial  impacts? 

5.  What  new  data  might  be  useful  to  enhance  the  analytical  capability  in  future 
cycles? 

6.  What  are  the  broader  challenges  in  determining  the  value  of  AISI  to  the 
performance  of  the  system  as  a  whole  and  how  can  these  be  overcome? 

1.2  Background 

Even  though  individual  AISI  projects  are  locally  initiated  and  are  designed  to  address  a 
wide  variety  of  issues,  a  common  goal  of  improved  student  learning  pervades  the 
program  (AISI  Cycle  3  Handbook,  p.  1).  This  is  most  evident  in  the  fact  that  almost  all 
projects  have  used  measures  of  student  learning  as  outcomes.  Baseline,  target  and 
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actual  outcomes  have  been  measured  and  reported  for  all  projects.  The  most  commonly 
used  measures  are  the  results  on  provincial  tests,  either  the  Provincial  Achievement 
Tests  (PATs)  or  the  Diploma  Examinations.  Many  projects  also  use  locally  determined 
achievement  measures,  including  standardized  tests  and  teacher-developed  measures. 
Additional  measures  include  surveys  of  teacher,  student  and  parent  satisfaction. 
However,  these  are  somewhat  less  common  and  are  typically  used  in  conjunction  with 
student  achievement  measures.  Since  the  PAT  and  Diploma  Exam  results  are  most 
pervasive  and  are  the  only  measures  that  can  be  directly  compared  across  projects  and 
project  types,  these  form  the  main  focus  of  this  report. 

A  major  issue  arising  from  the  analytical  work  to  date  is  the  impact  of  AISI  on  the 
school  system  in  Alberta  as  a  whole.  An  argument  can  be  made  that  an  initiative  of  the 
scope  of  AISI  (a  substantial  investment  of  funds  in  many  projects  over  a  long  time) 
should  be  expected  to  yield  some  impact  on  the  results  of  provincial,  and  possibly 
national  and  international  assessments."  The  validity  of  that  argument  and  the  question 
of  how  any  such  impacts  might  be  identified  will  be  examined  as  a  final  component  of 
this  project. 

1.3  The  AISI  Research  Design 

One  way  to  view  AISI  is  as  a  large  group  of  action  research  projects  on  school 
improvement.  Each  project  sets  its  own  target  outcomes  and  determines  what  it  wishes 
to  do  to  improve  these  outcomes.  However,  it  is  clear  from  much  of  the  work  already 
done  that  an  expectation  exists  for  provincial  impact,  and,  among  some  stakeholders, 
for  impact  on  provincial  achievement  measures.  This  leads  to  an  alternative  view  of 
AISI,  based  more  directly  on  quantitative  research  methods  and  design  principles.  From 
this  perspective,  AISI  can  be  thought  of  as  a  large  series  of  quasi-experiments,  with 
student  learning,  achievement  and  other  performance  indicators  as  dependent  variables 
(outcomes)  and  the  various  project  interventions  as  independent  variables  (or 
treatments).  The  goal  is  to  examine  the  impact  of  the  interventions  on  the  outcomes  and 
to  determine  which  kinds  of  interventions  yield  results  that  are  generalizable  to  a  larger 
population. 

Under  the  classic  quantitative  and  experimental  design  model,  determining  the  impact 
of  an  intervention  requires,  first,  that  an  impact  be  established  and,  second,  that  the 
impact  can  be  attributed  to  the  treatment  rather  than  to  extraneous  sources.  This  leads  to 
the  key  requirement  that  sources  of  influence  on  the  outcome,  other  than  the 
intervention  itself  (usually  called  extraneous  variables),  be  controlled.  In  "true 
experiments",  control  is  usually  exercised  by  the  use  of  random  assignment  of 
participants  (in  this  case  mainly  students  and  teachers)  to  treatment  and  control  groups 
and  comparing  the  changes  in  outcomes  over  time  for  the  two  (or  more)  groups.  This  is 


^It  is  important  to  note  that  Alberta  students  have  historically  performed  extremely  well  on  national  and 
international  assessments.  That  was  true  before  the  AISI  period  and  remains  true  today.  Because  school 
level  data  are  not  available  on  such  assessments,  it  is  not  possible  to  examine  any  impact  of  AISI  on 
performance  on  these  assessments. 
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often  referred  to  as  the  randomized  cUnical  trial"^.  The  key  design  point  is  not  that 
extraneous  variables  are  ehminated  but  that  randomization  allows  these  variables  to 
have  the  same  influence  on  both  groups  over  the  treatment  period,  so  that  any 
differences  between  the  groups  at  the  end  may  be  attributable  to  the  treatment. 

Although  the  true  experiment  is  regarded  by  many  as  the  "gold  standard"  for 
determining  if  a  treatment  is  effective,  in  practice,  very  few  large  scale  interventions  in 
school  systems  meet  the  design  requirements  of  an  experiment.  The  AISI  design,  which 
employs  measures  of  change  over  time,  but  no  explicitly  defined  control  group,  is  more 
typical.  Under  such  a  design,  changes  over  time  can  be  measured  but  these  changes 
cannot  be  attributed  directly  or  exclusively  to  the  intervention  because  other  events 
occurring  over  the  same  time  may  also  influence  the  outcome.  The  quasi-experimental 
nature  of  the  design  arises  because  the  intent  is  the  same  as  a  true  experiment. 
However,  some  of  the  conditions  of  a  true  experimental  design  are  met  while  others  are 
not. 

1.4  Analytical  Approaches  Used  in  Quasi-Experimental  Designs 

Under  a  quasi-experimental  design,  especially  when  a  large  number  of  projects  are 
available,  several  analytical  approaches  may  be  used  to  help  test  the  hypothesis  that 
outcomes  may  be  attributed  to  treatments.  The  main  alternatives  are: 

•  statistical  control 

•  replication 

•  time  series  analysis  (including  baseline  comparisons) 

•  differentiation 

Statistical  Control.  To  apply  statistical  control,  data  on  as  many  extraneous 
variables  as  possible  are  collected  as  part  of  the  study  and  techniques  such  as 
analysis  of  covariance  or  regression  analysis  are  used  to  separate  the  effects  of  these 
variables  from  the  "residual"  effect  of  the  treatment.  As  far  as  we  can  tell,  this  has 
not  been  done  in  the  AISI  projects  and  the  summary  data  files  available  do  not 
include  data  on  any  such  variables.  However,  since  a  baseline  measure  is  available, 
the  baseline  can  be  used  as  a  proxy  for  the  effects  of  extraneous  factors  occurring 
before  the  treatment  was  implemented  (though  not  those  occurring  during  the 
treatment  period).  This  allows  a  minimal  approach  to  statistical  control,  which  is 
used  in  this  study. 

Replication.  Replication  involves  repeated  application  of  the  same  treatment.  To 
the  extent  that  repeated  applications  yield  consistent  results,  the  probability  that  the 
results  are  caused  by  extraneous  variables  is  reduced.  Since  a  large  number  of 


The  term  "randomized  clinical  trial"  comes  from  medical  research.  The  term  "randomized  field  trial"  is 
less  commonly  used  but  is  closer  to  what  actually  happens  in  educational  experiments. 


11 


The  Learning  Mosaic 

projects  and  measures  within  specific  themes  are  available  in  AISI,  replication 
becomes  a  more  plausible  approach  to  analysis  -  although  we  must  still  remain 
cautious  that  even  projects  with  similar  themes  may  be  and  sometimes  clearly  are 
interpreted  and  implemented  in  very  different  ways  as  is  evident  in  the  cases 
described  in  the  accompanying  chapter  by  Sumara  and  Davis.  The  idea  is  to  look  for 
clusters  of  similar  projects  and  examine  the  consistency  of  results  across  these 
similar  projects.  If  these  results  can  also  be  supplemented  by  findings  from  similar 
studies  found  elsewhere  in  the  literature,  the  argument  for  treatment  effects  becomes 
more  plausible. 

The  meta-analytic  approach  used  in  existing  AISI  provincial  reports  is  a  good  example 
of  an  attempt  to  apply  the  replication  principle.  Meta-analysis  is  simply  a  way  of 
quantifying  the  average  results  over  a  number  of  studies.  Positive  average  results 
(judged  by  established  conventions  for  effect  sizes)  yield  evidence  of  a  treatment  effect 
that  may  be  thought  of  as  relatively  independent  of  the  design  differences  among 
specific  projects. 

Since  most  projects  are  repeated  over  three  years,  the  annual  repetitions  may  also  be 
considered  as  fairly  precise  replications  since,  presumably,  the  treatment  remains 
largely  the  same  from  year  to  year  although  even  here,  with  the  development  of 
knowledge  and  understanding  about  moving  from  novice  to  mature  professional 
learning  communities,  for  example,  one  would  expect  some  variations  and  even 
improvements  in  implementation  with  successive  "treatments"  in  any  one  case. 
Nonetheless,  in  this  study  the  year  by  year  results  will  be  examined  from  a  replication 
perspective. 

Time  Series  Analysis.  This  has  several  variations,  but  the  underlying  principle  is  to 
look  for  discontinuities  in  trends  plotted  over  time.  For  example,  if  student 
achievement  (by  whatever  measure)  is  steady  for  several  baseline  years,  then 
increases  with  the  application  of  a  treatment,  and  then  returns  to  its  original  state 
after  the  treatment  is  withdrawn,  this  would  constitute  evidence  (though  not 
conclusive  evidence)  of  a  treatment  effect.  Repeated  applications  and  withdrawals 
of  the  treatment,  with  similar  discontinuities,  would  strengthen  the  case  for  a 
treatment  effect.  A  steady  increase  in  achievement  before  and  after  a  treatment, 
while  desirable,  would  not  constitute  evidence  of  a  treatment  effect.  Indeed, 
fallacious  conclusions  are  sometimes  drawn  by  averaging  achievement  for  a  period 
before  and  a  period  following  the  treatment. 

Originally,  it  was  thought  that  the  relatively  long  time  frame  of  AISI  might  yield  a  basis 
for  time  series  analysis.  However,  a  review  of  the  data  indicates  that,  while  treatments 
are  typically  repeated  over  each  year  of  the  three-year  cycle,  the  data  are  compiled  on  a 
year-by-year  basis  and  cannot  be  tracked  cumulatively  for  the  same  students  over  the 
three  years.  There  is  thus  no  way  to  create  a  time  series  plot  of  change  over  the  three 
years.  Also,  there  is  no  evidence  on  effects  after  withdrawal  of  the  treatment,  as 
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students  have  not  been  followed  over  subsequent  years.  Despite  the  ability  to  compare 
baseline  and  actual  results,  the  AISI  design  does  not  lend  itself  to  a  true  time  series 
analysis. 

Dijferentiation.  Differentiation  is  the  complement  of  replication  in  the  sense  that 
the  goal  of  differentiation  is  to  examine  both  like  and  unlike  projects  designed  to 
meet  the  same  goals.  For  example,  the  effects  of  individualized  and  group  learning 
strategies  on  mathematics  achievement  may  ideally  be  examined  by  a  randomized 
clinical  trial  encompassing  both  treatments,  with  each  acting  as  a  control  against  the 
other.  In  the  absence  of  such  a  design,  each  treatment  may  be  applied  independently 
in  two  or  more  separate  studies.  In  this  case,  the  two  treatments  are  not  applied  to 
the  same  participants  or  to  different  randomly  assigned  participants  but  to  separate 
groups  that  may  be  more  or  less  equivalent,  depending  on  the  selection  process. 

When  differentiation  can  be  combined  with  replication,  the  prospects  for  making  valid 
comparisons  improve.  For  example,  if  we  had  three  projects,  each  involving  two 
teaching  strategies,  each  yielding  consistent  results  for  the  specific  intervention,  but 
different  results  across  interventions,  then  a  much  stronger  inference  about  the  effects  of 
each  intervention  can  be  made.  The  existence  of  a  large  number  of  projects  in  a 
program  such  as  AISI,  makes  it  possible  to  seek  replicated  examples  of  differentiated 
interventions,  bearing  in  mind  the  cautions  surrounding  implementation  expressed 
earlier. 

2.  Provincial  Overview 

This  section  gives  a  brief  overview  of  the  types  of  outcome  measures  used  in  AISI  and 
the  analytical  approach  used  in  the  provincial  reports.  Provincial  results  for  Cycles  1 
and  2  are  also  given.  These  have  been  drawn  directly  from  the  AISI  Provincial  Reports. 
A  parallel  analysis  was  conducted  for  the  first  two  years  of  Cycle  3  and  the  results 
presented  in  the  same  way  as  those  in  the  provincial  reports.  Results  are  given  for  the 
main  measures  used.  Specifically,  these  are  provincial  assessments  (PAT  and  Diploma 
Exam  performance),  local  assessments  (mainly  locally  administered  standardized  tests 
and  teacher  made  tests),  student  and  parent  surveys  and  teacher  surveys.  Each  of  these 
has  several  sub-sets,  which  are  examined  more  closely  in  later  sections. 

2. 1  Statistical  Note:  Effect  Size 

The  AISI  Provincial  Reports  use  as  the  main  indicator  of  effect  the  difference  between  a 
baseline  measure  (typically  an  average  for  the  three  years  prior  to  the  project)  and  an 
actual  project  result  (annual  or  over  three  years).  The  observed  difference  (for  example, 
the  difference  between  baseline  and  actual  on  a  particular  PAT  measure)  is  converted  to 
a  standardized  measure  called  an  "effect  size." 

An  effect  size  is  a  standardized  measure  of  the  effect  of  an  intervention,  which  permits 
comparisons  across  projects  with  similar  treatments  but  with  different  measures  of 
outcome.  The  use  of  effect  sizes  rather  than  the  more  conventional  tests  of  statistical 
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significance  facilitates  the  comparison  of  differences  across  measures  based  on  different 
scales  and  different  sample  sizes.  It  is  also  free  from  one  of  the  major  constraints  on 
statistical  significance  tests,  namely  that  statistical  significance  is  a  function  of  sample 
size.  For  interventions  with  large  sample  sizes,  as  is  the  case  for  many  AISI  themes  and 
strategies,  even  very  small  differences,  which  are  of  little  policy  importance,  can  be 
statistically  significant. 

Just  as  conventions  exist  forjudging  statistical  significance,  conventions  have  been 
adopted  to  indicate  whether  the  observed  effect  is  important  for  purposes  of  policy  or 
practice.  The  most  common  convention  is  that  proposed  by  Cohen  (1988)  as  follows: 

Effect  size  Interpretation 

<.20  no  effect 

.20-.49  small  effect 

.50-. 79  medium  effect 

.80  or  greater  large  effect 


AISI  uses  a  slightly  different  version,  which  counts  non-statistically  significant  or 
negative  results  as  no  effect  (Presumably  AISI  projects  are  never  designed  to  have 
negative  effects)  and  considers  effect  sizes  of  .01  to  less  than  .20  as  "minimal." 

2.2  Overview  of  Provincial  Results 

Figure  2.1  shows  the  provincial  results  for  Cycles  1  and  2,  and  the  first  two  years  of 
Cycle  3.  These  are  presented  to  give  context  for  the  re-analysis  and  to  allow  the 
differences  to  be  presented  in  the  remainder  of  this  report  to  be  clearly  identified  and 
accounted  for.  This  graph  also  facilitates  explanation  of  the  use  of  effect  sizes  rather 
than  statistical  significance  or  score  differences  as  the  basis  for  comparison. 
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Figure  2.1  Average  Effect  Sizes  for  AISI  Cycle  1,  Cycle  2  and  Cycle  3  (first  two  years) 
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The  effect  sizes  for  all  measures  other  than  provincial  assessments  are  mostly  in  the 
small  range  (.20  to  .49)  by  Cohen's  definitions.  The  provincial  assessment  results  show 
no  effect  by  Cohen's  convention  and  minimal  effect  by  the  AISI  convention.  There  is  a 
general  decline  in  effect  sizes  over  the  three  cycles  for  all  of  the  surveys,  a  decline  from 
Cycles  1  and  2  to  Cycle  3  for  local  assessments  and  a  decline  from  Cycle  1  but 
essentially  no  change  from  Cycle  2  to  Cycle  3  for  provincial  assessments. 

The  Cycle  2  Provincial  Report  noted  that  the  smaller  effect  sizes  for  provincial 
measures  and  larger  ones  for  local  assessments  are  to  be  expected,  as  the  latter  are  likely 
more  directly  related  to  what  the  projects  are  intended  to  accomplish.  Nevertheless,  the 
provincial  assessment  results  constitute  by  far  the  largest  single  source  of  data  and 
many  projects  explicitly  identify  their  goal  as  increasing  performance  on  these 
assessments.  Provincial  assessments  are  also  the  only  measures  that  are  common  across 
projects.  Despite  their  limitations,  this  makes  these  measures  more  useful  than  others 
when  attempting  to  determine  provincial  effects. 

3.  Cycles  2  and  3  Themes  and  Strategies 

3. 1  Cycle  2 

Although  AISI  projects  are  highly  varied,  it  is  possible  to  categorize  projects  into 
common  areas  based  on  subject,  type  of  student  served,  thematic  area,  teaching  strategy 
and  type  of  measures  used.  This  has  been  done  by  the  project  proponents  themselves, 
and  a  classification  system  has  been  developed  from  these  categories.  This  section 
examines  what  AISI  had  called  "themes"  and  "strategies"  for  the  Cycle  2  projects.  A 
parallel  analysis  for  the  first  two  years  of  Cycle  3  is  presented  in  the  next  section. 

Themes  and  strategies  for  which  20  or  more  projects  could  be  found  were  selected  for 
analysis.  A  few  combinations  of  themes  were  also  examined  (where  ten  or  more 
projects  could  be  found  for  the  combination)  as  a  way  of  investigating  the  possibility 
that  certain  cross-classifications  would  be  more  effective  than  others.  For  example, 
some  of  the  more  common  Cycle  2  project  types  involved  the  use  of  professional 
learning  communities  within  a  particular  teaching  or  learning  strategy  such  as  problem 
solving  or  cooperative  learning.  Such  combinations  were  identified  and  examined 
separately. 

Average  effect  sizes  over  the  duration  of  the  project  for  all  projects  within  a 
theme/strategy  were  computed.  Confidence  intervals  were  then  computed  across 
projects  within  a  theme,  strategy  or  combination.  A  theme,  strategy  or  combination  was 
considered  to  have  a  statistically  significant  effect  if  the  confidence  interval  around  the 
average  effect  size  did  not  overlap  zero. 

Table  2. 1  is  an  attempt  to  summarize  these  results  by  theme,  strategy  and  combination 
and  by  the  various  measures  used.  The  abbreviations  "Acc"  and  "Ex"  in  the  table  refer 
to  the  percentage  of  students  reaching  the  Acceptable  Standard  and  the  Standard  of 
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Excellence  on  the  Provincial  Achievement  Tests,  the  Grade  12  Diploma  Exams  and 
Local  achievement  measures.  The  "Other"  category  for  local  measures  refers  to  results 
such  as  the  percentage  of  students  meeting  grade  level  expectations,  percentage 
graduating,  average  standardized  test  scores  and  other  outcomes.  For  the  survey 
measures,  the  average  effect  sizes  were  based  on  percentage  response  to  questionnaire 
items  on  satisfaction  or  similar  scales.  The  use  of  effect  sizes  throughout  allows 
comparisons  to  be  made  across  these  diverse  measures. 

Most  of  the  effect  sizes  for  the  two  types  of  PAT  measures  are  quite  small.  While  there 
are  variations  across  themes,  the  variability  within  themes  is  generally  as  great  as  or 
greater  than  the  effect  sizes  themselves.  This  indicates  that  there  is  not  much 
consistency  among  projects  within  specific  themes.  Indeed,  in  only  a  few  cases  are  the 
effects  of  a  theme  statistically  significantly  positive,  as  most  of  the  error  ranges  overlap 
the  zero  point  on  the  scale.  No  theme  effect  is  consistently  positive  across  both  the 
acceptable  standard  and  the  standard  of  excellence. 

Effect  sizes  for  the  Diploma  Exams  are  generally  larger  than  those  for  the  PATs  and 
most  are  statistically  significantly  positive.  Also,  the  effects  are  larger  for  the  standard 
of  excellence  than  for  the  acceptable  standard,  although  there  is  considerable  variation 
within  themes.  What  is  most  interesting  about  this,  however,  is  the  relatively  small 
differences  in  effect  across  themes,  especially  for  the  standard  of  excellence.  This 
suggests  that  something  other  than  the  AISI  themes  is  functioning  to  yield  positive 
results  throughout.  As  later  results  will  indicate,  these  results  reflect  a  general  upward 
trend  in  the  percentage  of  students  meeting  the  Standard  of  Excellence  over  the  Cycle  2 
period. 
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Local  measures  at  the  acceptable  standard  are  available  for  a  few  themes.  Four  of  these 
-  differentiated  instruction,  professional  development,  professional  learning 
communities  and  alternative  delivery  -  show  statistically  significant  effects.  Effects  are 
larger  and  more  variable  across  themes  on  the  "other"  standard.  However,  the  detailed 
results  show  that  effects  are  generally  highly  variable  within  themes.  This  category  of 
measure  encompasses  a  variety  of  different  standards,  so  there  is  little  consistency  in 
what  this  category  means,  especially  across  projects.  For  this  and  other  reasons  (such  as 
potential  bias  in  local  measures),  it  is  difficult  to  make  a  judgment  on  the  meaning  of 
these  effects. 

Average  effects  sizes  for  parent  surveys  are  mostly  small  (.20  or  less  according  to  the 
detailed  results),  though  many  are  statistically  significant  within  themes.  Some  larger 
effects  (in  the  .20  to  .30  range)  are  apparent  for  the  student  surveys.  It  is  worth  noting 
that  some  themes  which  might  be  described  as  "student  centered,"  such  as  learning 
styles,  cooperative  learning,  individualized  instruction  and  mentoring  yield  among  the 
larger  effects  as  measured  by  student  surveys,  even  though  these  do  not  stand  out  in  the 
achievement  measures. 

The  table  shows  many  significant  positive  effects  for  the  teacher  growth  and  teacher 
satisfaction  measures.  The  detailed  results  indicate  that  the  largest  effects  are  those  for 
teacher  growth.  Most  of  these  effects  reach  the  "moderate"  level  by  Cohen's  criteria. 
Relative  to  the  effect  sizes,  there  is  also  somewhat  less  variation  within  themes  for 
teacher  growth.  There  might  be  some  expectation  that  professional  development  themes 
would  be  particularly  attractive  to  teachers.  Indeed,  the  professional  learning  community 
theme  does  yield  one  of  the  highest  average  effect  sizes.  However,  once  the  within- 
theme  variation  is  considered,  this  theme  does  not  stand  out  as  exceptional. 

It  is  not  possible  to  identify  from  the  detailed  results  themes  that  stand  out  as 
contributing  most  or  least  to  teacher  growth.  The  teacher  growth  effects  are  essentially 
all  much  larger  than  those  for  other  measure  types.  This  reinforces  the  earlier  point  that 
other  factors  may  be  at  play.  The  possibility  of  a  generic  effect  stemming  from  the 
resources  made  available  as  well  as  the  attention  shown  towards  teachers  and  their 
professional  judgment  is  a  strong  one  here.  Although  one  might  expect  this  to  show  up 
in  teacher  surveys  as  well  as  the  growth  measures,  the  effects  for  teacher  surveys  do  not 
stand  out  as  particularly  large. 

3.2  Cycle  3 

The  themes  and  strategies  used  in  AISI  Cycle  3  were  quite  similar  to  those  found  in 
Cycle  2.  In  particular,  differentiated  instruction,  professional  development  and 
professional  learning  communities  remained  major  themes.  However,  there  are  a  few 
notable  differences.  Assessment  projects  were  much  more  common  in  Cycle  3,  with 
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"assessment  for  learning"  being  classified  as  a  strategy  rather  than  a  theme. There  was 
a  slightly  greater  concentration  of  projects  in  Cycle  3,  with  several  areas  that  were  found 
in  Cycle  2  not  having  sufficient  measures  in  Cycle  3  to  meet  the  selection  criteria  for 
this  analysis.  On  the  other  hand,  a  few  areas,  specifically  blended  structure,  individual 
programming  and  high  school  completion  were  added  in  Cycle  3. 

The  Cycle  3  measures  also  paralleled  those  in  Cycle  2  with  one  exception.  A  category 
labelled  "provincial  satisfaction  measures"  was  added  to  allow  for  the  use  of  a  battery 
of  measures  administered  by  the  province  to  measure  satisfaction  with  the  school 
system  on  the  part  of  various  stakeholders,  including  students,  teachers  and  parents. 
The  different  versions  of  the  satisfaction  measures  were  not  differentiated  in  the  AISI 
data  base,  so  only  combined  results  can  be  reported  here. 

Table  2.2  shows  the  summary  results  for  the  first  two  years  of  Cycle  3.  The  effects  here 
are  consistently  somewhat  more  positive  than  were  found  for  Cycle  2,  especially  for  the 
Acceptable  Standard.  The  detailed  results  again  show  effect  sizes  to  be  quite  small 
(typically  <.  10  but  with  some  variation). 

Results  for  the  Diploma  Exams  show  few  statistically  significant  effects.  The  general 
trend,  as  shown  in  the  detailed  results,  was  towards  slightly  negative  gains.  This 
pattern  is  quite  different  from  that  found  in  Cycle  2,  especially  for  the  standard  of 
excellence,  where  the  results  generally  have  shifted  from  small  positive  to  no  effect 
over  all  themes  and  strategies. 

Relatively  few  results  were  available  for  locally  developed  measures  in  Cycle  3  and 
most  of  those  that  are  available  show  no  significant  effect.  Again,  this  is  different  from 
what  was  found  in  Cycle  2,  where  most  effects  were  positive. 


Assessment  was  also  classified  as  a  theme  in  Cycle  3  but  this  category  almost  completely  overlapped 
the  assessment  for  learning  strategy.  The  latter  was  used  in  the  analysis  because  more  projects  and 
measures  were  identifiable. 
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As  before,  the  results  for  the  student  and  parent  surveys  tend  to  be  positive  across 
themes  and  strategies.  However,  the  detailed  results  show  considerable  variation  within 
themes  and  strategies.  Three  strategies  -  mentoring,  cooperative  learning  and  small 
group  instruction  -  stand  out  in  the  detailed  results  as  having  greater  and  more 
consistent  effects  across  the  measures  than  others.  All  of  these  also  showed  significantly 
positive  effects  for  these  same  measures  in  Cycle  2.  Differentiated  instruction  and 
assessment  for  learning  also  show  consistent  effects  across  measures.  However,  this  is 
at  least  partly  an  artifact  of  the  large  number  of  projects  for  each  measure,  resulting  in 
smaller  within-theme  errors.  All  of  these  also  showed  significantly  positive  effects  for 
these  same  measures  in  Cycle  2. 

The  provincial  satisfaction  surveys  formed  a  distinct  measure  in  Cycle  3.  Most  of  the 
effect  sizes  available  for  this  measure  are  marginally  positive  indicating  that  satisfaction 
with  the  system  as  a  whole  was  increasing. 

3.3  Cycle  2  and  Cycle  3  Summary 

The  picture  for  Cycle  2  and  3  effects  by  theme/strategy  and  measure  is  complex  and 
difficult  to  summarize  in  a  concise  way.  What  is  clear  is  that  the  effects  within 
particular  themes  or  strategies  are  not  highly  consistent.  There  is  somewhat  greater 
consistency  within  measures  across  themes  and  strategies.  However,  that  is  not 
particularly  helpful  in  determining  whether  some  themes  or  strategies  are  more  effective 
than  others. 

One  way  to  create  a  concise  summary  is  to  look  for  effects  that  are  consistently  positive 
over  the  two  cycles.  This  can  be  made  more  explicit  by  setting  some  effect  size  criterion 
and  determining  whether  this  criterion  is  met  for  themes  and  strategies  within  measures. 
In  this  case,  a  relatively  liberal  criterion  was  adopted,  based  on  the  idea  that  the  mean 
effect  size  for  a  particular  theme/strategy  and  measure  can  be  considered  statistically 
significant  if  its  confidence  interval,  as  represented  by  the  error  bars  on  the  graphs,  does 
not  overlap  zero  for  either  cycle. 

Table  2.3  shows  these  consistent  effects.  It  is  again  evident  that  the  surveys  yield  the 
most  significant  effects,  further  reinforcing  the  point  that  the  results  are  more  measure- 
specific  than  theme/strategy  specific.  However,  under  this  criterion,  four  areas  - 
professional  development,  assessment,  cooperative  learning  and  workshops  -  show 
significant  PAT  effects.  Professional  development  and  workshops  overlap  by  about 
40%,  so  these  should  not  be  considered  as  independent  effects. 
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4.  Correlational  and  Multivariate  Analysis 

4. 1  Correlations  Over  Time 

Most  Cycle  2  projects  were  repeated  in  each  of  the  three  years  of  the  cycle,  while  two 
repetitions  were  available  for  Cycle  3.  One  simple  way  to  test  whether  the  results  for 
particular  projects  are  consistent  over  time  is  to  correlate  the  results  over  the  years  for 
which  a  particular  project  is  repeated.  In  summary  correlations  were  found  to  be 
consistently  positive,  as  expected,  and  that  their  magnitudes  were  relatively  large.  Since 
correlation  coefficients  may  be  interpreted  directly  as  effect  sizes,  by  Cohen's  criteria 
most  of  the  effects  were  at  least  medium  and  many  were  large.  This  supports  the  view 
that,  despite  maturations  in  understanding  and  in  approaches  to  implementation,  it  is  still 
reasonable  to  treat  repetitions  of  a  project  over  time  as  replications. 

4.2  Correlations  of  Outcomes  with  Baseline 

Correlations  of  the  AISI  outcomes  with  the  baseline  measures  yielded  the  expected  result 
with  correlations  averaging  more  than  .80.  This  may  be  taken  as  an  indication  that  the 
measures  are  highly  reliable  and  that  AISI  projects  are  no  exception  to  the  general  pattern 
in  pretest/posttest  studies  where  previous  behaviour  is  typically  the  strongest  predictor  of 
later  behaviour.  What  is  more  important,  however,  is  that  the  correlations  between 
baseline  and  effect  sizes  were  generally  negative.  This  suggests  that  the  results  may  be 
subject  to  what  is  known  as  statistical  regression.  In  effect,  this  means  that  projects 
applied  to  students  with  lower  baseline  scores  are  likely  to  yield  positive  effects  while 
those  with  higher  baseline  results  are  likely  to  yield  negative  effects.^  When  a  measure  is 
repeated  (as  in  comparing  baseline  to  later  outcome),  the  regression  effect  results,  on 
average,  in  an  increase  in  the  scores  at  the  lower  end  of  the  distribution  and  a  decrease  in 
those  at  the  higher  end.  This  is  unrelated  to  any  treatment  effect  but  may  be  confounded 
with  the  treatment  effect,  especially  when  looking  at  results  that  are  particularly  strongly 
positive  or  negative. 

A  second  effect,  which  seems  related  to  regression  but  differs  in  principle,  is  what  is 
known  as  a  "ceiling  effect".  In  common  sense  terms,  this  relates  to  the  expectation  that  it 
would  more  difficult  to  achieve  gains  for  students  who  are  already  at  the  high  end  of  the 
achievement  scale  than  for  those  at  the  lower  end.  There  is  a  link  between  regression  and 
the  ceiling  effect  in  that  both  can  serve  to  reduce  outcome  scores  at  the  high  end  and 
increase  them  at  the  low  end.  However,  these  effects  are  not  logically  related.  All  other 
things  being  equal,  the  regression  effect  will  actually  reduce  outcome  values  for  those  at 
the  high  end  whereas  the  ceiling  effect  will  more  likely  result  in  smaller  gains  but  not 
actual  reduction. 


^  A  commonly  used  illustration  of  statistical  regression  is  the  height  of  children  relative  to  their  parents. 
Children  of  tall  parents  will  tend  to  be  tall  because  of  heredity  but  will  tend  to  be  shorter  than  their  parents 
because  of  other  factors  affecting  parents'  height  which  are  not  present  in  the  offspring.  The  opposite  is 
true  for  the  children  of  shorter  parents.  The  same  phenomenon  applies  to  any  quasi-experiment  where 
subjects  are  selected  from  the  high  or  low  ends  of  the  baseline  distribution  and  the  change  over  time  is 
measured. 
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In  order  to  examine  this  point  more  closely,  the  distributions  of  baselines  by  project  and 
measure  were  divided  into  five  equal  groups,  or  quintiles.  The  mean  effect  sizes  were 
then  computed  for  each  quintile.  These  results  may  be  summarized  as  follows: 

•  The  general  pattern  was  one  of  positive  effects  for  low  baseline  projects  and  zero 
or  negative  effects  for  high  baseline  projects. 

•  The  results  for  the  PATs  and  the  Diploma  Exams  were  essentially  symmetrical, 
with  the  positives  for  low  baseline  projects  being  mostly  offset  by  negatives  for 
the  high  baseline  projects.  This  is  consistent  with  the  small  overall  effects  for 
these  measures,  and  suggests  that  the  treatment  effect  is  too  small  to  be  detected, 
once  the  regression  effect  is  taken  into  account. 

•  For  the  survey  measures,  almost  all  of  the  effects,  for  both  high  and  low  baseline 
projects  were  positive,  suggesting  a  treatment  effect  that  is  independent  of  the 
regression  effect.  However,  the  pattern  of  higher  effects  for  low  than  for  high 
baseline  projects  was  as  pronounced  for  the  surveys  as  for  the  achievement 
measures. 

Even  though  these  results  point  to  the  likelihood  that  regression  effects  are  strong 
contributors  to  the  results,  it  is  not  a  simple  matter,  under  a  quasi-experimental  design,  to 
separate  the  treatment  from  the  regression  effects^  or  the  regression  from  the  ceiling 
effect.  The  fact  that  the  average  effect  remains  positive  for  most  measures,  and  that  the 
low  and  high  quintile  results  are  not  generally  symmetrical,  indicates  that  something 
other  than  regression  is  at  play.  Also,  an  argument  can  be  made  that  higher  gains  on  the 
part  of  those  with  the  lowest  starting  point  is  a  desirable  outcome  in  any  event.  However, 
this  is  not  particularly  useful  when  the  goal  is  to  separate  treatment  effects  from 
regression  effects. 

4.3  A  Multiple  Regression  Model 

As  mentioned  earlier,  statistical  techniques  may  sometimes  be  used  to  control  for 
extraneous  variables  that  may  be  confounded  with  the  treatment  effects.  The  possibilities 
for  using  statistical  controls  on  the  AISI  data  are  limited  because  the  data  base  does  not 
include  background  information  on  the  schools,  teachers  and  students  taking  part  in  the 
projects.  However,  a  model  can  be  developed  in  which  the  baseline  values  are  used  as  a 
proxy  for  all  of  the  possible  external  effects  which  may  exist.  The  assumption  here  is  that 
student  demographic  and  socioeconomic  backgrounds,  school  characteristics,  teaching 
strategies,  and  other  pre-treatment  factors  that  might  affect  the  outcome  are  embodied  in 
the  baseline.  Controlling  for  the  baseline  also  controls  for  both  the  regression  and  ceiling 
effects.  While  this  is,  at  best,  a  rough  approximation,  a  model  which  controls  for  the 
baseline  can  help  shed  some  light  on  the  extent  to  which  treatments  have  an  effect 
independent  of  many  extraneous  background  factors. 


^  In  a  true  experiment,  the  regression  effect  is  not  eliminated  but  is  controlled  by  virtue  of  random 
assignment  of  participants  to  the  treatments.  That  is,  this  effect  appears  about  equally  in  the  treatment  and 
control  groups. 
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Two  models,  one  with  effect  size  as  the  outcome  and  one  with  the  mean  actual  score, 
were  run  for  each  measure  for  which  sufficient  data  were  available.  The  results  may  be 
summarized  as  follows: 

•  As  expected,  the  baseline  is  a  strong  positive  predictor  of  the  actual  outcome. 
Those  who  start  at  a  high  baseline  tend  to  remain  high  and  vice  versa  for  those 
who  start  low,  regardless  of  any  other  effects. 

•  The  baseline  is  generally  a  strong  negative  predictor  of  the  effect  size.  This  again 
offers  evidence  of  ceiling  effects  and/or  regression  effects. 

The  important  question  is  whether  any  of  the  themes  or  strategies  show  effects  that 
remain  positive  after  controlling  for  the  baseline.  Taking  the  results  over  the  two  types  of 
outcomes  and  the  two  cycles  shows  that  many  of  the  effects  that  were  positive  in  the 
initial  analysis  become  negative  once  the  baseline  is  controlled.  This  suggests  that  some 
of  the  observed  effects  for  particular  measures  are  artifacts  of  the  baselines  for  projects 
involving  these  treatments. 

A  few  themes  and  strategies  were  found  to  be  significant  predictors  of  both  effect  size  and 
actual  outcome  within  a  cycle.  These  effects  are  summarized  in  Table  2.4.  With  baseline 
controlled,  it  is  more  plausible  to  consider  these  as  treatment  effects.  Most  of  these  effects 
are,  however,  measure-specific.  Also,  none  shows  a  consistent  effect  across  the  two  cycles. 


Table  2.4  Consistent  Theme  and  Strategy  Effects  for  Effect  Size  and  Actual  Outcomes  by  Measure 


Measure 

Cycle  2 

Cycle  3 

PAT  acceptable 

Technology  integration  (+) 

Small  groups  (+) 
Professional  development  (+) 
Professional  learning 
communities  (+) 

PAT  excellence 

Early  intervention  (-) 

Small  groups  (+) 
Professional  development  (+) 
Professional  learning 
communities  (+) 

DE  Acceptable 

Learning  styles  (-) 

High  school  completion  (-) 

DE  Excellence 

Local  achievement  measures 
acceptable 

One  on  one  (-) 
Problem  solving  (-) 
Cooperative  learning  (-) 
Balanced  literacy  (-) 
Workshops  (+) 

Learning  styles  (+) 

Local  measures  "other" 

PLC  (-) 

Cooperative  learning  (-) 
Mentoring  (+) 

Parent  survey 

Alternative  delivery  (-) 

Mentoring  (+) 

Student  survey 

Problem  solving  (-) 
Assessment  (-) 
Cooperative  learning  (+) 

Mentoring  (+) 

Teacher  growth 

Differentiated  instruction  (-) 

Teacher  satisfaction 

Alternative  delivery  (-) 
Problem  solving  (-) 
One  on  one  (-) 

High  school  completion  (-) 
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5.  AISI  Results  Relative  to  Provincial  Trends 

One  issue  that  has  arisen  in  discussion  of  the  small  effect  sizes  for  provincial  measures  is 
whether  provincial  averages  have  changed  over  time  and  whether  it  is  desirable  (or  even 
possible)  to  adjust  the  AISI  results  for  any  such  change.  This  section  addresses  this  issue 
for  the  PATs,  for  which  provincial  average  results  are  available. 

5.7  PAT  Provincial  Overview 

PAT  provincial  results  over  all  grades  (3,  6  and  9)  and  subjects  (language  arts,  mathematics, 
science  and  social  studies)  are  based  on  detailed  analyses  and  may  be  summarized  as 
follows: 

•  There  was  slight  growth  in  provincial  average  performance  on  both  standards 
during  the  Cycle  2  baseline  period  from  2001  to  2003. 

•  The  provincial  averages  were  quite  stable  over  the  Cycle  2  years,  2004  to  2006. 

•  Taking  2004  to  2006  as  the  baseline  for  Cycle  3,  the  provincial  average  for  the 
acceptable  standard  declined  in  the  Cycle  3  years,  while  the  average  for  the 
standard  of  excellence  increased  slightly.^  This  suggests  greater  variability  of 
results  in  these  years. 

5.2  AISI  Averages  Compared  to  Provincial  Averages 

The  main  question  of  interest  is  whether  the  AISI  results  follow  the  same  pattern  as  the 
provincial  results.  It  is  difficult  to  determine  this  for  the  overall  results  across  all  PAT 
measures  because  the  mix  of  subjects  and  grades  in  AISI  is  not  the  same  as  that  for  the 
province  as  a  whole.  For  that  reason,  the  analysis  was  conducted  by  subject  and  grade  for 
English  language  arts  and  mathematics  only. 

The  results  for  the  PAT  acceptable  standard  showed  that  the  AISI  baselines  were 
generally  lower  than  the  provincial  baselines,  especially  in  Cycle  3.  This  suggests  that 
AISI  projects  may  have  been  targeted  to  some  extent  at  students  or  schools  that  had  been 
performing  below  the  provincial  average  in  years  prior  to  the  project.  For  the  most  part, 
the  AISI  trend  lines  were  similar  to  the  provincial  trend  lines,  suggesting  that  AISI 
projects  generally  did  not  break  the  general  provincial  pattern.  However,  there  were 
some  notable  exceptions  to  this.  The  most  obvious  was  for  Grade  3  English  language  arts, 
where  AISI  students  started  at  a  relatively  low  baseline  average  and  continued  to  improve 
in  2007  and  2008  even  in  the  face  of  an  overall  provincial  decline  in  those  two  years. 
The  pattern  is  similar,  though  not  so  pronounced,  for  English  language  arts  in  the  other 
two  grades.  The  performance  of  AISI  students  in  grade  3  mathematics  showed 
improvement  in  2007  despite  provincial  decline  but  declined  with  the  further  provincial 
decline  in  2008. 


The  percentage  at  the  acceptable  standard  actually  refers  to  "at  least  acceptable"  so  this  includes  those  at 
the  standard  of  excellence.  A  reduction  in  the  percentage  at  the  acceptable  level  thus  implies  that  more 
students  are  below  that  level. 
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Comparable  figures  for  the  PAT  standard  of  excellence  showed  a  general  tendency  for 
AISI  students  to  have  been  above  the  provincial  average  in  Cycle  2  but  below  in  Cycle  3. 
Beyond  this,  the  general  pattern  is  for  the  AISI  trend  to  follow  the  provincial  trend. 
While  this  may  be  partially  an  artifact  of  the  fact  that  the  reported  AISI  numbers  are 
likely  higher  than  the  actual  number  of  students  involved  in  AISI  projects,  this  is  not  the 
sole  reason,  since  the  AISI  and  provincial  means  were  different  in  most  cases. 

5.3  Can  the  AISI  Effects  be  Adjusted  for  Provincial  Trends? 

In  principle,  to  avoid  confounding  of  AISI  effects  with  provincial  trends,  any  changes 
over  time  on  provincial  measures  found  for  AISI  projects  should  be  adjusted  for  these 
changes  in  overall  provincial  results  over  the  same  time  period.  In  practice,  this  proved 
to  be  problematic  for  several  reasons,  including  variations  in  the  differences  across 
subjects  and  grades,  differences  in  the  mix  of  subjects  and  grades  for  AISI  and  the 
province  as  a  whole  and  problems  with  the  number  of  students  reported  as  participating 
in  the  PATs  within  projects.  The  data  available  were  therefore  not  judged  to  be  precise 
enough  to  permit  any  direct  adjustment  of  AISI  annual  results  for  provincial  trends. 
However,  this  could  be  done  if  a  more  precisely  matched  student  level  data  set  were 
available. 

6.  Summary  and  Conclusions 

6.1  Interpretation  of  the  Results 

These  results  support  the  conclusion  that  positive  change  has,  indeed  occurred  over  time 
and  across  AISI  projects  and  measures.  However,  the  AISI  design  does  not  allow  this 
change  to  be  attributed  unambiguously  to  AISI  as  a  whole  or  to  specific  themes  or 
strategies  within  AISI.  In  particular,  the  effect  of  AISI  is  confounded  with  a  statistical 
regression  effect.  It  is  also  not  possible  to  clearly  separate  AISI  effects  from  other 
changes  that  may  have  occurred  over  time.  This  is  an  inherent  limitation  of  the  before  and 
after  design;  a  limitation  that  cannot  be  overcome  by  post  hoc  statistical  methods.  While 
analyses  such  as  those  conducted  here  can  point  to  potential  alternative  explanations  for 
the  outcome,  they  cannot  definitively  distinguish  the  AISI  treatment  effects  from  other 
possible  sources  of  change. 

The  fact  that  effects  are  larger  for  surveys  than  for  achievement  measures  is  indicative  of 
a  possible  Hawthorne  or  halo  effect.  The  Hawthorne  effect  was  first  noticed  in  classic 
studies  in  organizational  theory  in  the  1920s  when  worker  productivity  increased  as  a 
result  of  various  interventions  -  even  ones  that  led  to  deteriorations  in  physical  working 
conditions  -  because  the  workers  were  responding  favourably  to  positive  attention  in  an 
environment  where  little  of  it  had  been  accorded  to  them  before.  That  is,  AISI  may  well 
be  perceived  as  effective  by  various  stakeholders  because  of  the  attention  and  positive 
emotions  of  respect  and  recognition  that  are  shown  to  participants,  without  there  actually 
being  any  impact  on  student  learning.  There  is  a  distinct  possibility  that  the  additional 
resources  available,  and  their  use  for  various  desirable  purposes  in  schools,  may  be  the 
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cause  of  the  relatively  large  effect  sizes  for  the  teacher  measures  in  particular.  Again, 
this  cannot  be  positively  identified  as  the  cause  of  the  survey  effects,  any  more  than  can 
the  actual  AISI  treatments. 

While  some  might  argue  that  a  Hawthorne  or  halo  effect  is  as  good  as  a  real  one, 
especially  if  the  goal  is  to  engender  positive  attitudes,  it  is  not  clear  that  this  was  the 
purpose  of  AISI  as  a  school  improvement  program.  The  fact  that  provincial  measures 
were  the  most  common  outcome  indicators  used,  and  that  the  goals  of  many  projects 
were  stated  in  terms  of  improvement  on  these  measures,  attests  to  the  importance  of 
achievement  outcomes  in  the  AISI  design.  The  marginal  effects  on  these  measures  should 
be  a  source  of  concern  -  either  in  terms  of  failure  to  have  impact  on  these  measures,  or  in 
terms  of  whether  these  measures  are  the  most  appropriate  ones  or  have  been  selected 
simply  because  they  are  readily  available. 

The  general  absence  of  differential  results  across  themes  and  strategies  may  be  interpreted 
in  two  possible  ways.  First,  it  is  possible  that  all  of  the  themes  and  strategies  adopted  have 
previously  been  shown  to  be  effective,  and  hence  there  is  no  reason  to  believe  that  one 
theme  would  be  more  effective  than  another.  The  logical  extension  of  that  argument  is 
that  it  may  not  matter  exactly  what  is  done  as  long  as  it  is  consistent  with  principles  or 
practices  that  have  previously  been  demonstrated  to  be  effective. 

The  second,  and  conflicting,  argument  is  that  it  is  implausible  to  think  that  all  themes  or 
strategies  would  be  equally  effective  and  that  generic  themes,  such  as  professional 
development,  might  be  expected  to  have  smaller  effects  than  more  student-oriented 
strategies  such  as  cooperative  learning  or  differentiated  instruction.  In  this  second  case, 
the  absence  of  strong  differential  effects  across  themes  and  strategies,  combined  with  the 
small  overall  effect  sizes  for  provincial  achievement  measures,  is  evidence  that  AISI  has 
had  little  impact  on  student  attainment  as  measured  by  these  indicators. 

It  is  sometimes  argued  that  an  appropriate  way  to  examine  effectiveness  is  to  look  at 
"outlier  projects,"  specifically  those  with  the  largest  positive  or  negative  effects.  There  is 
some  indication  from  Chapter  6  of  the  Cycle  2  Provincial  Report  that  this  approach  has 
been  used  in  AISI.  Unfortunately,  this  technique  is  rendered  largely  useless  because  of 
the  existence  of  what  appears  to  be  a  large  regression  effect.  It  is  not  plausible  to  explain 
why  virtually  all  treatments  are  highly  effective  for  low  baseline  students  and  highly 
ineffective  for  high  baseline  students,  other  than  by  a  regression  effect. 

It  is  also  possible  that  the  AISI  results  are  being  influenced  by  external  events  that  are 
occurring  within  public  education  in  Alberta,  which  cannot  be  accounted  for  in  the 
analysis.  It  is  clear,  for  example,  that  Alberta  has  experienced  in  recent  years  an  influx  of 
new  residents,  from  both  other  provinces  and  through  immigration.  This  means  new 
students  and  new  teachers,  and  likely  greater  student  mobility  within  the  province  than  is 
being  experienced  elsewhere.  None  of  these  effects  are  being  accounted  for  in  the  AISI 
design  or  recorded  in  the  data  base  in  ways  that  allow  their  effects  to  be  separated  from 
AISI  treatment  effects. 
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6.2  Limitations  of  the  AISI  Research  Design 

It  is  important  to  recognize  that  AISI  was  conceptuaHzed  as  a  school  improvement 
initiative  and  not  a  research  initiative.  While  innovation,  risk-taking  and  experimentation 
were  all  inherent  in  AISI,  the  program  was  not  designed  to  meet  the  strict  scientific 
conditions  required  to  establish  causal  conclusions.  At  the  same  time,  the  desire  to  show 
effects  on  a  provincial  scale  has  led  to  attempts,  including  this  one,  to  determine  if  the 
program  has  had  positive  effects  on  student  learning. 

Unfortunately,  it  is  not  possible  to  impose  an  experimental  design  on  the  program 
retroactively  and  post  hoc  methods  cannot  substitute  for  such  a  design.  However,  these 
methods,  particularly  replication,  differentiation  and  statistical  control  can  bring  us  some 
way  beyond  the  direct  baseline-actual  comparisons.  The  more  detailed  analysis 
conducted  here  suggests  that  extraneous  regression  effects  as  well  as  possible  Hawthorne 
effects  are  plausible  rival  explanations  for  the  AISI  results. 

A  further  design  issue  has  to  do  with  definition  and  implementation  of  the  treatments. 
Under  an  experimental  design,  each  treatment  would  have  to  be  carefully  defined 
operationally,  individuals  would  have  to  be  trained  to  implement  the  treatment  in  as 
uniform  a  way  as  possible,  and  processes  would  typically  have  been  developed  to 
monitor  implementation.  None  of  this  is  easy  in  settings  as  complex  as  schools  and 
classrooms,  and  it  is  not  surprising  that  the  AISI  design  did  not  include  such  elements. 
However,  the  absence  of  such  information  precludes  us  from  saying  anything  about 
fidelity  of  implementation  or  from  estimating  the  amount  of  error  in  the  outcomes  that  is 
due  to  variation  in  implementation. 

6.3  Limitations  of  the  Measures 

A  further  concern  is  whether  the  measures  used,  particularly  the  PATs  and  Diploma 
Exams,  are  appropriate  indicators  of  AISI  outcomes.  In  a  typical  experimental  or 
quantitative  input-output  design,  the  specific  outcomes  expected  from  the  treatments 
would  be  defined  and  measures  of  these  outcomes  devised.  It  is  not  uncommon  for 
educational  improvement  outcomes  to  be  defined  in  terms  of  standardized  test  results. 
However,  in  this  case,  it  seems  likely  that  the  PATs  and  Diploma  exams  were  selected 
more  for  their  convenience  and  universal  availability  than  for  their  direct  relationship  to 
the  treatments  of  interest.  There  are  indications  that  the  results  for  these  and  other 
measures  included  in  the  data  base  are  not  matched  directly  to  the  students  involved  in 
particular  projects.  Also,  most  students  in  Alberta  public  schools  seem  to  be  involved  in 
one  or  more  AISI  projects  and  the  duration  of  their  involvement  may  vary  with  student 
mobility  or  grade  placements.  Although  the  PATs  are  administered  to  individual  students 
only  every  three  years,  results  are  available  at  the  school  or  district  level  every  year.  In 
the  absence  of  student  level  data,  it  is  impossible  to  determine  to  whom  the  reported 
results  apply,  whether  the  same  students  are  in  a  project  for  one,  two  or  three  years,  or 
whether  students  are  involved  in  multiple  projects. 

It  might  be  argued  that  local  student  learning  measures  are  better  than  the  provincial 
measures  because  they  can  be  more  specifically  tailored  to  the  projects.  It  is  noted  that 
many  of  these  measures  are  also  standardized  tests  of  various  kinds.  What  is  not  known  is 
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whether  these  were  selected  to  create  a  fit  to  the  project  or  if  these  were  also  selected  for 
convenience.  In  any  event,  the  great  variety  of  these  measures  makes  it  difficult  to  judge 
their  appropriateness  or  to  compare  outcomes  across  measures.  Technically,  the  use  of 
effect  sizes  addresses  the  comparability  problem  by  standardizing  the  measure  of  change. 
However,  this  does  not  help  in  interpreting  what  the  change  means. 

As  for  the  surveys,  it  has  already  been  pointed  out  that  the  results  on  these  measures  may 
plausibly  be  interpreted  as  pointing  to  a  Hawthorne  or  halo  effect.  Some  might  argue  that 
improved  results  on  these  measures  are  desired  outcomes  in  their  own  right.  This  is  a 
matter  of  what  outcomes  are  valued  by  the  Alberta  system.  However,  as  already  noted, 
there  is  no  clear  way  to  distinguish  between  AISI  effects,  regression  effects  and  general 
changes  over  time  and  halo  effects  on  these  outcomes. 

6.4  Data  Base  Quality  Issues 

Systematic  data  collection,  with  a  focus  on  outcomes,  has  been  part  of  the  AISI  design 
from  its  inception.  The  AISI  quantitative  data  base,  along  with  other  descriptive 
documentation  on  projects,  provides  a  rich  source  of  information  on  the  program, 
extending  over  nearly  a  decade.  More  generally,  AISI  has  been  well  documented  in 
provincial  reports  at  the  end  of  each  cycle,  research  reviews,  annual  conferences  and 
symposia  and  presentations  at  national  and  international  conferences. 

Nevertheless,  this  analysis  has  revealed  several  limitations  of  the  data  base,  from  the 
point  of  view  of  statistical  analysis  of  outcomes.  The  main  ones  are  related  to  the  quality 
and  suitability  of  the  measures  as  indicated  above.  Others  are  more  specific  to  what  data 
are  gathered  and  how  the  data  base  is  compiled. 

The  data  base  consists  of  data  aggregated  to  the  level  of  average  scores  on  individual 
measures  within  each  project.  The  raw  data  from  which  these  aggregates  have  been 
developed  consist  of  achievement  scores  for  individual  students  and  attitude/satisfaction 
scores  for  students,  teachers  and  parents.  However,  the  individual  scores  are  not  recorded 
in  the  data  base.  Much  of  the  variation  in  individual  scores  is  lost  in  the  aggregation. 
This  precludes  the  use  of  techniques  such  as  multilevel  analysis,  which  can  separate 
within-project  variation  from  between  project  variation.  Furthermore,  in  the  case  of  the 
PAT  and  Diploma  Exam  measures,  and  some  others  as  well,  this  limits  the  analysis  to 
percentages  meeting  a  specific  criterion  (mainly  the  acceptable  standard  and  the  standard 
of  excellence)  rather  than  to  average  scores  or  other  measures  of  central  tendency  and 
variation. 

While  the  difficulty  in  compiling  and  managing  an  individual  level  data  base  is 
appreciated,  this  is  now  the  norm  in  most  large  scale  assessments  and  other  projects  of 
this  nature.  Individual  raw  data  are  almost  certainly  available  in  electronic  data  base 
form  somewhere  in  the  system.  What  is  needed  is  a  way  to  bring  these  data  together. 
This  should  not  be  particularly  difficult  for  provincial  measures,  as  the  raw  data  base 
exists  at  a  provincial  level.  What  is  needed  is  a  way  to  match  individual  students  with 
projects.  Since  a  universal  student  ED  does  exist  in  Alberta,  this  would  be  a  matter  of 
matching  the  project  ID  to  the  individual  student. 
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A  student  level  data  base  would  permit  an  analysis  of  the  level  of  participation  of 
individual  students  in  projects  and  would  ensure  that  the  data  are  aggregated  for 
participants  rather  than  for  all  students  in  a  school  or  district,  as  seems  now  to  be  the  case 
for  many  provincial  measures.  This  would  also  permit  an  analysis  of  the  duration  of 
student  participation,  student  mobility  and  other  factors,  participation  in  multiple  projects 
and  other  aspects  of  student  exposure  to  projects. 

Finally,  the  data  base  does  not  include  data  on  student  or  school  characteristics.  Again,  a 
considerable  amount  of  such  data  must  exist  in  other  data  bases  within  the  Alberta  public 
school  system.  Certainly  school  characteristic  data  (e.g.,  school  size,  grade  levels, 
location,  school  level  socioeconomic  status,  characteristics  of  the  student  body,  school 
level  performance  on  provincial  measures,  etc.)  are  available  and  could  likely  be  fairly 
easily  merged  with  the  AISI  data  base.  A  data  file  does  exist  on  participating  schools  in 
each  project.  If  the  characteristics  of  the  participating  schools  could  be  merged  with  that 
file,  this  would  be  a  first  and  major  step  in  creating  a  disaggregated  AISI  data  base.  This 
could  be  done  without  creating  a  complete  new  data  base  and  is  recommended  for  the  full 
Cycle  3  provincial  analysis. 

6.5  Research  Design  Issues 

It  is  perhaps  unrealistic  to  argue  that  AISI  as  a  whole  should  be  transformed  into  a  large 
set  of  randomized  clinical  trials.  However,  it  seems  reasonable  to  argue  that  at  least  some 
of  the  larger  scale  themes  and  strategies,  such  as  professional  learning  communities, 
differentiated  instruction  or  assessment  for  learning  should  be  investigated  with  more 
rigorous  research  designs.  Although  many  of  these  areas  have  been  widely  researched, 
the  dearth  of  large  scale  clinical  trials  limits  the  confidence  with  which  we  can  attribute 
improvements  in  student  learning  to  these  treatments.  Taking  this  approach  might  make 
it  possible  for  Alberta  to  join  the  ranks  of  the  few  jurisdictions  that  have  made  a  major 
contribution  to  the  quality  of  evidence  in  education  through  policies  that  allow  and 
support  randomized  clinical  trials. 

Added  to  this,  it  would  be  helpful  if  Alberta  Education  were  to  clarify  whether 
improvement  on  provincial  measures  is  an  explicit  goal  of  AISI.  If  not,  then  it  may  have 
to  be  acknowledged  that  gains  on  provincial  measures  are  not  the  chief  priority  and  that 
other  outcomes  should  receive  greater  emphasis. 

6.6  Policy  Issues 

Several  policy  issues  arise  from  this  analysis.  First,  there  is  the  obvious  question  of  what 
outcomes  are  most  important  and  particularly  of  the  relative  weight  to  be  placed  on 
provincial  versus  local  achievement  measures  compared  to  attitude  or  satisfaction 
surveys.  If  the  main  concern  is  that  local  schools  and  jurisdictions,  and  their  student, 
teacher  and  parent  stakeholders  are  satisfied,  then  AISI  provides  some  evidence  that  that 
is  the  case  -  which  can  be  alternatively  interpreted  as  a  Hawthorne  or  halo  effect  of 
educators  receiving  attention  and  being  given  a  voice  in  classroom  and  school  decision- 
making by  policy  makers.  If,  on  the  other  hand,  the  concern  is  with  enhancing  the 
already  high  performance  of  Alberta  students  on  provincial  measures  (and  by  extension 
to  national  and  international  measures),  then  it  is  difficult  to  argue  from  the  results  of  this 
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study  that  AISI  has  had  any  noticeable  effect.  Between  and  beyond  these  two  positions  is 
the  consideration  as  to  whether  AISFs  apparent  effects  on  teacher  growth  and  student 
engagement  or  attitudes  are  worthwhile  in  their  own  right  provided  they  do  not  have 
negative  effects  on  measured  student  attainment. 

A  related  issue  is  that  of  value  being  received  for  the  resources  invested  in  AISI.  While 
this  issue  cannot  be  fully  addressed  from  the  results,  a  couple  of  useful  points  might  be 
made.  For  example,  AISI  could  be  justified  as  a  large  scale  research  program,  designed 
to  determine  if  particular  themes  or  strategies  are  effective  in  improving  performance  on 
valued  outcomes.  While  the  investment  in  AISI  is  large  relative  to  most  research 
programs  in  education,  it  is  not  large  relative  to  research  in  the  natural  and  medical 
sciences.^  It  is  also  not  a  particularly  large  component  of  operating  expenditures  on 
public  schooling  in  Alberta.  In  fact,  the  annual  average  of  about  $70  million  since  the 
inception  of  AISI  amounts  to  about  two  percent  of  that  total. 

In  a  more  ideal  world,  and  certainly  under  an  optimal  research  design,  one  would  expect 
to  find,  among  such  a  large  array  of  themes  and  strategies,  some  that  are  much  better  than 
others  that  could  be  recommended  for  wider  adoption.  The  fact  that  few  interventions 
could  be  identified  as  especially  effective  or  ineffective  suggests  either  that  all  AISI 
themes  and  strategies  are  of  equal  value  or  that  the  value  of  any  specific  theme  or  strategy 
does  not  matter  as  long  as  there  is  a  net  gain  to  the  system.  This  takes  us  back  to  the  idea 
that  of  the  impact  of  a  systemic  Hawthorne  effect. 

6. 7  Recommendations 

Even  within  the  current  quasi-experimental  approach,  a  number  of  measures  might  be 
taken  which  could  improve  the  scope  and  quality  of  the  data  available  for  secondary 
analysis.  Most  of  these  have  already  been  mentioned  and  are  simply  reiterated  here. 

•  A  student  level  data  base  is  needed,  which  includes  baseline  and  outcome  data  on 
individual  students,  as  well  as  background  information  on  these  same  individuals, 
and  linking  this  data  base  to  existing  provincial  data  bases.  This  is  key  to 
improving  the  ability  to  apply  statistical  control  techniques  to  distinguish 
treatment  effects  from  other  influences  on  outcomes. 

•  More  generally,  all  data  should  be  recorded  in  the  data  base  at  the  lowest  level  of 
aggregation  at  which  the  data  are  collected. 

•  Systems  should  be  developed  for  monitoring  the  implementation  of  treatments  or 
innovations  and  the  quality  of  the  data  provided. 

•  Clearer  documentation  is  needed  on  the  content  of  local  and  survey  measures. 


As  a  rough  comparison,  the  Alberta  Research  Council  has  an  annual  operating  budget  of  $93  million  plus 
about  $50  million  in  grants  and  contracts.  This  compares  to  $70-75  million  annually  for  AISI. 
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•  Analysis  is  required  on  the  vaHdity  of  the  measures  being  used  in  relation  to 
project  goals  and  designs.  More  specifically,  a  clear  decision  is  needed  on 
whether  provincial  measures  should  continue  to  be  emphasized,  and/or  others 
developed  instead/as  well. 

•  Further  analysis  is  required  on  student  and  school  characteristics  as  possible 
contributors  to  the  results.  Some  of  this  can  be  done  by  judicious  merging  of  the 
AISI  data  base  with  other  data  bases  maintained  by  Alberta  Education.  Other 
work  along  these  lines  would  require  enhancement  of  the  data  base,  as  indicated 
above. 

The  following  recommendations  apply  to  the  broader  policy  issues  of  AISI  goals  and  the 
link  between  research  goals  and  more  general  innovation  and  school  improvement  goals. 

•  A  clearer  statement  is  needed  on  the  research  function  of  AISI  relative  to  its  other 
purposes,  particularly  to  its  purpose  as  a  stimulator  of  innovation. 

•  Some  of  the  major  themes  and  strategies  within  AISI  should  be  investigated  using 
a  randomized  clinical  trial  design.  This  would  address  the  confounding  of  AISI 
effects  and  extraneous  effects  inherent  in  the  existing  design  and  would  permit  the 
program  to  contribute  more  directly  to  the  broader  literature  on  the  effectiveness 
of  specific  innovations. 
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Chapter  3:  Using  Complexity  Science  to  Study  the  Impact  of  AISI  on 
Cultures  of  Education  in  Alberta 

By  Dennis  Sumara  and  Brent  Davis 

1.  Introduction  and  Overview 

1.1  Orienting  Questions 

This  investigation  was  oriented  by  the  following  queries,  originally  posed  in  October 
2008  by  Alberta's  Deputy  Minister  of  Education,  Keray  Henke: 

1.  What  is  the  value  of  the  Alberta  Initiative  for  School  Improvement  (AISI)? 

2.  Why  couldn't  jurisdictions  do  this  anyway  (without  AISI)? 

3.  Would  the  values  of  AISI  continue  without  funding? 

4.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

We  were  invited  to  bring  a  'complexity'  reading  to  these  questions  -  in  brief,  to  study  the 
manners  in  which  AISI  might  have  affected  educational  cultures  within  different  school 
districts,  their  perceived  benefits  and  difficulties,  and  whether  such  influences  might  be 
self-sustainable. 

1.2  Research  Frame:  Complexity  Thinking 

Complexity  thinking  has  arisen  over  the  last  half  century.  In  education,  it  has  been  used 
in  relation  to  areas  such  as  neurological  process,  interpersonal  dynamics,  and  global 
ecology.  Complexity  research  is  characterized  more  in  terms  of  what  one  investigates 
than  how  one  investigates.  We  operationally  define  complexity  research  as  the  study  of 
learning  systems.  Any  adaptive  system  that  exists  within  a  vibrant  context  can  be 
understood  as  a  "learner."  Necessary  traits  of  complex  systems  compiled  by  complexity 
researchers  that  are  also  relevant  to  this  study  include: 

•  Self-Maintenance  -  complex  systems/unities  adapt  and  adjust  on  the  fly  in 
response  to  other  dynamic  agents  and  evolving  circumstances; 

•  Self-Amplification  -  the  behaviors  of  complex  unities  are  better  characterized  in 
terms  of  feedback  loops  (that  self-amplify  or  self-dampen)  than  in  predictable 
linear-causal  terms; 

•  Harmonization  of  Internal  Redundancy  and  Internal  Diversity  -  complex  systems 
do  not  operate  in  balance  -  indeed,  a  stable  equilibrium  implies  death  for  a 
complex  system.  Rather,  the  internal  dynamics  of  complex  unities  are  dynamic 
harmonies,  such  as  an  ongoing  dialectic  of  samenesses  and  differences  among 
subsystems  within  a  grander  system; 
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•  Level-Jumping  -  owing  to  the  nested  structures  of  complex  systems,  it  is  possible 
(and  often  necessary)  to  distinguish  and  select  those  levels  of  an  organizational 
system  that  are  most  relevant  for  any  issue. 

•  Decentralized  Network  Structure  -  the  'fingerprint'  of  a  complex  unity  is  a 
fractal-like  structure  of  nodes  of  subsystems  clustering  in  larger  nodes,  which  in 
turn  cluster  into  larger  nodes  -  each  giving  rise  to  new  patterns  of  activities  and 
new  rules  of  behavior; 

This  latter  quality  is  of  particular  significance  to  this  research  and  relates  to  network 
theory. 

1.3  Network  Types 

Within  network  theory,  four  categories  of  networks  have  been  identified  (see  Fig.  3.1). 
Each  has  a  specific  structure  with  advantages  and  disadvantages. 


Figure  3.1.  Four  Types  of  General  Network  Structures  (drawn  on  identical  sets  of  dots): 
centralized,  decentralized,  fragmented,  and  distributed. 

The  centralized  network  has  a  hub  through  which  all  relationships  (e.g.,  flow  of 
information,  channeling  of  resources)  are  mediated.  Advantages  are  efficient 
communication  and  resource  distribution.  However,  it  is  only  as  robust  and  only  as 
flexible  as  the  central  hub. 

At  the  other  extreme,  a  distributed  network  is  characterized  by  tight  and  extensive  local 
connectivity,  but  no  large-scale  systemic  connectivity.  This  structure  has  the  advantage  of 
being  very  robust.  However,  distribution  and  communication  is  very  inefficient  -  and,  by 
consequence,  phenomena  structured  this  way  are  highly  resistant  to  change. 

A  decentralized  network  comprises  many  centers.  Nodes  in  such  networks  tend  to  be 
decentralized  networks  themselves.  This  structure  combines  reasonably  efficient 
communication  with  a  reasonably  robust  structure,  enabUng  considerable  flexibility  and 
high  adaptability. 

Because  the  decentralized  network  structure  is  associated  with  life  and  learning,  it  has  a 
certain  susceptibility  to  contexts.  For  example,  a  decentralized  system  can  be  'forced'  to 
take  on  a  more  centralized  organization  through  increased  stress  or  constraint  (e.g., 
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sudden  limitations  on  resources,  removal  of  certain  freedoms).  Conversely,  shifts  toward 
more  distributed  structures  can  be  triggered  by  removing  stresses  and  constraints  (e.g., 
overabundant  resources,  removal  of  accountability  measures).  As  well,  for  all 
decentralized  networks,  there  is  an  ever-present  possibility  of  decay  into  a  fragmented 
network,  in  which  the  grander  coherence  fails  through,  for  example,  the  loss  of  a  layer  of 
'connective  tissue'  such  as  a  shared  purpose  or  reliable  intermediaries. 

Triggering  the  emergence  or  recovery  of  a  decentralized  network  structure  is  more 
complex  than  destroying  one.  Figure  3.2  illustrates  some  possible  starting  places  in  such 
efforts.  Decentralized  networks  must  be  in  constant  disequilibrium.  There  must  be 
stressors  (both  familiar  and  surprising)  that  compel  them  to  adapt/learn.  Such  stressors 
should  not  be  seen  as  causes  for  change,  but  as  occasions  for  transformation. 

CENTRALIZED 


DISTRIBUTED    .*  * 


FRAGMENTED 

Figure  3.2  Some  triggers  for  movement  among  different  network  structures  within  social  systems 

These  possibilities  map  out  how  people  and  ideas  can  network.  We  applied  these 
possibilities  to  investigating  communications  and  relationships  and  the  emergence  or  not 
of  robust  and  productive  learning  systems  within  three  districts,  with  particular  reference 
to  AISL  This  analysis  also  revealed  insights  into  external  relationships  and  networks  with 
a  grander  learning  system  at  the  provincial  level. 

1.4  Methodology 

The  unit  of  analysis  or  "learning  system"  in  this  study  was  the  school  district.  Three  were 
selected  in  consultation  with  the  School  Improvement  Branch  of  Alberta  Education:  one 
in  a  large  urban  setting,  one  'rural,'  and  the  third  in  a  smaller  urban  setting.  Being 
attentive  to  the  temporal  frames  of  social  and  cultural  systems,  this  report  is  thus 
organized  around  three  descriptive  case  studies,  oriented  by  an  ethnographic  attitude 
towards  the  meanings  and  mindsets  of  each  district's  culture,  and  informed  by  network 
theory. 
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Fieldwork  was  accomplished  through  a  two-day  visit  to  each  district.  These  visits 
comprised  information  sessions,  focus-group  interviews,  and  individual  interviews  - 
involving  administrators,  teachers,  school  board  officials,  parents,  students,  and  other 
community  members.  In  each  district,  meetings  were  conducted  in  two  schools  and  at  the 
school  board  office.  We  aimed  to  discern  the  networked  structures  in  each  district,  how 
these  networks  might  have  arisen,  and  how  they  enabled  and  constrained  possibilities  for 
AISI  within  the  district.  We  were  specifically  interested  in  how  people  were  or  were  not 
connected  to  one  another,  how  resources  and  information  are  distributed  and  accessed, 
and  how  ideas  and  innovations  circulate. 

7.5  Structure  of  the  Chapter 

Our  data  and  interpretations  are  presented  as  "portraits"  of  the  three  districts  and  what 
each  perceives  its  work  to  be.  Using  data  gathered  during  our  site  visits,  we  describe  the 
mindset  that  structures  each  district's  ways  of  conceptualizing  its  work  and  its 
understanding  of  learners  and  teachers.  We  explain  how  each  mindset  informs  how 
different  roles  and  relationships  are  enacted,  how  these  enactments  contribute  to  the 
district  mindset,  and  how  all  of  these  influence  the  ways  school  innovation  and  school 
improvement  (through  AISI)  is  being  understood  and  enacted.  Each  portrait  is  organized 
by  the  following  five  questions: 

1.  How  does  the  District  conceptualize  its  work? 

2.  How  does  this  conception  of  work  influence  this  District's  implicit  conceptions  of 
learning  and  learners? 

3.  How  do  these  conceptions  of  learning  and  learners  contribute  to  enacted  roles  and 
relationships  in  this  District? 

4.  How  do  these  enacted  roles  and  relationships  contribute  to  the  developing  and 
maintaining  of  the  organizational  mindset  that  guides  this  District? 

5.  How  does  this  organizational  mindset  influence  the  ways  AISI  is  understood, 
conceptualized,  and  operationalized  within  this  District? 


2.  Pathways  School  District:  The  work  of  the  District  is  learning. 

2.1  How  does  Pathways  conceptualize  its  work? 

During  a  focus  group  meeting,  the  Superintendent  explained:  "Learning  is  the  work. 
Things  are  always  changing  in  our  District."  This  emphasis  on  learning  was  evident  in 
every  interview  we  conducted.  School  Board  members  described  their  role  as  "learning 
how  to  pay  attention  to  what  our  students  and  teachers  need  to  enhance  their  learning." 
Teachers  continually  expressed  the  importance  of  career-long  learning  as  a  way  to  ensure 
that  students'  learning  would  be  enhanced.  One  teacher  explained,  "When  I  first  came  to 
this  District  I  thought  I'd  be  here  for  only  a  year  or  two.  What  has  kept  me  here  for  many 
years  has  been  the  extraordinary  opportunities  to  learn  that  have  been  provided." 

Enabling  this  emphasis  on  learning  is  a  seemingly  paradoxical  relationship  between  site- 
based  management  and  collective  decision-making.  While  schools  are  given  considerable 
autonomy  with  budgets  and  school  management,  the  explicit  linking  between  and  among 
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schools  through  structured  and  organized  practices  of  District-wide  consultation  and 
collaboration  have  helped  to  create  conditions  for  this  District  to  continually  monitor  and 
adjust  its  activities  and  priorities. 

Coupled  to  these  structural  and  organizational  features  was  the  often-expressed  belief  that 
ongoing  learning  and  the  change  that  accompanies  learning  is  enhanced  by  a  willingness 
to  engage  with  ideas  and  practices  that  are  not  entirely  known  or  familiar.  As  the  District 
AISI  Coordinator  explained,  "We  know  the  importance  of  taking  risks  -  to  take  risks  and 
flounder.  We  know  that  if  we're  willing  to  do  that  a  lot  of  good  will  come  of  it."  A  school 
principal  commented,  "In  order  to  be  an  effective  learner  you  have  to  be  willing  to  be  in  a 
continuous  state  of  fluctuation."  Commenting  on  the  importance  of  supporting  innovative 
projects  funded  by  AISI,  a  School  Board  member  stated,  "Sometimes  it's  like  cooking 
soup  that  turns  out  differently  than  what  was  thought.  I  would  like  to  see  risk  being 
increased  rather  than  calmed  down." 

2.2  How  does  this  conception  of  work  influence  the  Pathway's  implicit  conceptions  of 
learning  and  learners? 

Change  and  learning  are  considered  to  be  synonymous,  and  it  is  understood  that  change 
cannot  occur  without  some  degree  of  difficulty  and  uncertainty.  As  one  administrator 
explained,  "There  are  tensions  between  capacity  building  and  measuring,  tensions 
between  creating  the  conditions  to  risk  and  to  innovate."  Another  administrator 
explained:  "We  send  groups  of  teachers  to  conferences  and  we  ask  them  to  bring  those 
ideas  back  here  and  use  them.  . . .  We  are  prepared  to  take  risks  with  what  we've  learned." 

All  schools  have  used  AISI  funding  to  schedule  time  for  teachers  to  meet  and  work  in 
groups.  Lead  teachers  from  each  school  meet  at  a  District  level  to  share  ideas.  The  work 
of  learning  and  learners,  then,  is  considered  to  emerge  from  the  work  of  collaborative 
connectivity.  One  teacher  explained,  "We  know  that  if  three  of  us  decide  to  do  it,  it  takes 
the  riskiness  out  of  it.  We  can  push  ourselves  further  individually  because  we  have 
support."  Another  stated,  "Our  administrators  encourage  us  to  try  new  approaches.  If  they 
don't  work,  we  know  we  can  try  something  else."  Effective  learners  and  learning,  then, 
emerge  from  strong  relationships  among  learners. 

Teachers  and  administrators  in  this  District  tended  to  use  collective  pronouns  (we/us) 
rather  than  personal  pronouns  when  discussing  learners  or  learning  (including  their  own 
learning).  Administrators  regarded  the  creation  of  these  strong  relationships  as  one  of 
their  primary  responsibilities.  One  District  administrator  explained,  "The  more 
relationships  you  can  build  in  leadership,  the  better  chance  you  will  have  with  being 
supported  with  change."  The  work  of  learning  is  understood  in  this  District  as  lifelong 
and,  therefore,  the  learning  that  can  be  noticed  and  measured  in  schools  is  viewed  with 
healthy  skepticism.  A  school  principal  suggested,  "It  takes  more  than  three  years  for  a 
system  to  change."  Another  explained,  "I  don't  think  we'll  notice  the  effects  of  our  AISI 
projects  in  three  years.  If  we  track  these  kids,  we'll  notice  the  most  significant  changes  in 
20  years." 
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2.3  How  do  these  conceptions  of  learning  and  learners  contribute  to  enacted  roles  and 
relationships  in  the  Pathways  District? 

Key  people  have  become  'hubs'  in  the  District's  decentrahzed  network,  carrying  an 
important  historical  perspective  on  how  developments  in  the  District  may  or  may  not  link 
to  emerging  challenges  and  opportunities.  The  Superintendent  explained,  "There  is  an 
openness  to  our  conversations.  [The  AISI  Coordinator]  is  the  conscience  of  the  admin 
team.  He  is  the  person  people  look  to  in  the  system  to  say  difficult  things  that  need  to  be 
said." 

People  and  systems  are  fluidly  organized  in  ways  that  allow  specialized  knowledge  to 
move  to  the  fore  as  needed  to  contribute  to  the  emerging  directions  being  developed  in 
the  District.  As  a  result,  the  District  embodies  an  understanding  that  leaders  and 
leadership  must  shift  in  an  adaptive  and  distributed  way  as  the  system  adapts  and 
changes.  In  the  words  of  one  school  principal  "My  role  is  to  say  'yes.'  If  you  believe  that 
power  is  meant  to  be  given  away,  then  you  say  'yes'  to  things  even  if  you  can't  do  them 
or  personally  control  them."  Teachers  confirmed  this  sensibility.  "There  is  no  clear  leader 
[of  our  AISI  projects].  There  are  a  number  of  them  who  take  responsibility  for  different 
things  at  different  times."  This  emphasis  on  building  and  maintaining  relationships 
emerges  because,  as  one  principal  put  it,  "We  are  building  leadership  capacity  through 
collaboration.  It's  being  done  by  building  relationships  and  trust." 

This  approach  to  decentralized  leadership  is  supported  by  the  District's  conception  of  the 
District's  work  as  learning,  and  by  the  need  for  learners  and  learning  to  be  supported  by 
nested  layers  of  supportive  communities.  The  AISI  Coordinator  for  the  District  described 
it  in  this  way:  "We  have  well-developed  networks  of  people  throughout  the  District  who 
know  how  to  work  together.  We  have  had  considerable  training  in  collaboration  and  in 
how  to  develop  and  use  a  professional  community  model  of  learning." 

2.4  How  do  these  enacted  roles  and  relationships  contribute  to  the  developing  and 
maintaining  of  the  organizational  mindset  that  guides  the  Pathways  District? 

The  boundaries  around  institutions  and  people  in  different  roles  are  porous.  Teachers, 
staff,  administrators,  and  consultants  know  one  another  and  rely  on  these  relationships  to 
assist  the  ongoing  work  of  learning.  One  teacher  knew  "80%  of  the  300  teachers  in  the 
District."  Another  observed,  "We  are  not  closed  off  in  any  way  in  this  District."  The 
Superintendent  stated,  "I  can  go  to  any  school  and  tell  you  by  name  who  is  in  each 
school.  There  are  many  opportunities  for  people  to  get  to  know  me  and  one  another." 

This  well-developed  culture  of  relationship  building  supported  the  ways  new  ideas  were 
infused  into  the  system  -  often  through  informal  interactions.  One  new  teacher  explained, 
"You  can  bring  your  ideas  to  anyone,  anytime.  Like  you're  standing  in  line  at  the  Tim 
Norton's  and  [names  Superintendent  by  first  name]  is  there  -  and  I  can  give  him  my  ideas 
right  there."  Another  added,  "Everyone  seems  to  have  a  voice:  parents,  community 
members,  teachers  -  everyone.  And  our  School  District  accepts  all  of  that.  You  can  talk 
about  anything  to  anyone." 
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The  importance  of  connecting  and  collaborating  seems  to  have  instilled  into  the  district  a 
robust  regard  for  difference  and  diversity  -  with  an  implicit  understanding  that  these 
qualities  create  conditions  for  enhanced  systemic  learning.  A  school  board  member,  AISI 
Coordinator  and  teacher  made  the  same  point:  "We  have  a  respect  for  diversity,"  "we 
can't  all  walk  down  the  same  road  together  at  the  same  time." 

2.5  How  does  this  organizational  mindset  influence  the  ways  AISI  is  understood, 
conceptualized,  and  operationalized  within  the  Pathways  District? 

Pathways  had  longstanding  processes  and  structures  that  facilitated  communication, 
connection,  and  collaboration  in  schools,  between  schools  and  local  communities,  and 
between  and  among  schools  and  the  District  office.  These  values  of  seeing  the  primary 
work  of  the  District  as  learning  have  been  catalyzed  by  AISI  opportunities.  The 
Superintendent  explained,  "I  see  AISI  as  the  spark  in  the  network."  This  helps  to  explain 
the  depth  and  breadth  of  understanding  of  AISI  projects  across  this  District  community, 
and  why  AISI  initiatives  were  not  disconnected  from  the  District's  overall  work. 

Pathways  has  been  able  to  leverage  AISI  resourcing  to  enhance  the  learning  potential  of 
the  entire  District.  Pathways  seems  to  believe  that  the  best  way  to  improve  schools  and 
student  learning  is  to  develop  and  support  a  robust  decentralized  network  of 
communication,  collaboration,  and  decision-making,  all  of  which  are  attuned  to  lifelong 
learning  for  staff,  teachers,  administrators.  School  Board  members,  and  members  from 
the  community.  In  creating  this  network  of  learning,  the  District  seems  to  have 
accommodated  considerable  diversity  of  projects,  ideas,  and  opinions  yet  also  maintain  a 
robust  internal  coherence  in  terms  of  emergent  mission  and  vision.  As  several  teachers 
explained:  "[AlSI-supported  structures  and  initiatives]  have  allowed  us  to  develop  a 
common  language.  We  are  basically  all  on  the  same  page." 

2.6  Summary 

Among  the  core  elements  of  the  Pathways  District's  self-narrative  are  connectivity,  a 
commitment  to  shared  work,  an  appreciation  of  the  inevitability  of  change,  and  support 
for  innovation.  Prior  to  AISI,  considerable  energy  and  resources  had  already  been 
committed  in  support  of  professional  development.  Across  the  District,  a  culture  had 
emerged  in  which  new  and  "disorienting"  ideas,  processes,  practices,  and  structures  could 
be  incorporated  and  tested  without  compromising  the  integrity  of  any  one  individual  or 
school.  In  particular,  the  District  seemed  able  to  take  advantage  of  diversity  -  both  pre- 
existing and  newly  introduced  -  of  people,  experience,  practice,  belief,  and  ideas. 

This  long  history  of  valuing  communicative  connectivity  as  a  way  to  enrich  learning 
across  all  levels  of  the  District  meant  that  Pathways  was  able  to  use  AISI  funding  to 
amplify  what  it  perceived  as  its  primary  work:  finding  ways  to  enhance  learning  for  all 
students  and  teachers.  Most  AISI  resources  have  been  used  to  further  improve  webs  of 
connectivity  and  communication  by,  for  example,  leveraging  AISI  dollars  against  other 
funding  to  provide  release  time  for  teachers  to  meet.  As  a  learning  culture,  Pathways  has 
learned  to  think  in  groups  and  to  be  aware  of  how  new  ideas  emerge  from  these  collective 
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learning  experiences.  In  the  process,  Pathways  is  able  to  conceive  of  itself  as  a 
knowledge-producing  system,  and  not  merely  a  knowledge-disseminating  system. 
A  vital  aspect  of  this  knowledge-producing  character  arises  in  the  District  having 
managed  to  tap  and  support  internal  and  external  sources  of  diversity.  New  ideas  are 
deliberately  drawn  in  by  subgroups,  while  structures  are  in  place  to  recognize  and  utilize 
local  expertise.  This  powerful  use  of  diversity  is  greatly  enabled  by  the  emergence  of  a 
"shared  vocabulary"  -  a  conscious  recognition  of  the  importance  of  redundancies  on  the 
level  of  collective  action. 

Pathways'  emphasis  on  learning  enables  it  to  exist  comfortably  in  an  ongoing  state  of 
disequilibrium.  The  District  confronts  its  challenges  through  a  decentralized  structure,  in 
which  key  hubs  -  places,  structures,  and  personalities  -  operate  to  ensure  connectivity  of 
stakeholders,  flow  of  information,  and  archiving  of  experience.  These  hubs  are  often 
organized  around  key  people,  including  the  current  Superintendent  and  AISI  Coordinator. 
However,  consistent  with  a  decentralized  structure,  many  others  have  emerged  as  key 
hubs  and  nodes  in  the  system's  network,  leading  classroom-,  school-,  and  district-wide 
initiatives. 


3.  Hearthstone  School  District:  The  work  of  the  district  is  service, 

3.1  How  does  Hearthstone  conceptualize  its  work? 

Within  Hearthstone,  a  conception  of  "service  as  the  work"  appeared  to  drive  an 
uncompromising  work  ethic  relative  to  learning  and  teaching,  with  particular  attention  to 
ensuring  that  the  very  best  ideas  and  practices  would  be  known  by  as  many  teachers  as 
possible.  An  Associate  Superintendent  explained,  "We  have  been  on  a  journey  of  making 
cultural  change  in  the  District.  We  now  have  in  place  a  robust  bottom-up/top-down 
collaborative  process,  which  has  really  helped  us  to  use  our  AISI  funds  productively." 

Over  the  years,  this  District  has  established  a  robust  layer  of  specialist  consultants  who 
act  as  the  "clearinghouse"  for  educational  research  and  best  practices  for  teachers.  Some 
of  these  work  out  of  the  District  office,  while  others  work  as  specialist  teachers  in 
schools.  Funded  in  large  part  through  AISI,  this  layer  of  specialist  support  has  become  an 
important  way  for  this  District  to  connect  people  to  new  ideas  from  research  and  to 
ensure  that  these  ideas  are  integrated  into  teaching  practice.  As  one  consultant  explained, 
"I  spend  two  to  three  days  a  week  coaching  teachers  with  new  ideas  and  practices  that 
I've  learned." 

3.2  How  does  this  conception  of  work  influence  Hearthstone's  implicit  conceptions  of 
learning  and  learners? 

In  this  District,  learning  is  the  work  of  each  individual.  The  work  of  individual  learners 
should  have  discernible  and  measurable  products.  One  senior  administrator  explained, 
"We  have  graphed  our  students'  progress  over  time,  and  we  have  seen  the  changes  on  the 
PATs."  A  diversity  of  approaches  to  learning  and  knowledge  dissemination  is  supported 
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by  attention  to  developing  specialization.  Strong  interdisciplinarity  has  been  achieved 
through  enhancement  of  subject  area  and  issue/practice  (e.g.,  inclusive  education, 
pastoral  counseling  services)  specializations. 

This  willingness  to  devote  considerable  time  and  money  to  specializations  is  supported 
by  what  a  deep  and  abiding  respect  for  the  autonomy  of  individuals,  who  become  linked 
in  community  through  shared  beliefs  and  values  in  this  District,  and  who  are  also  linked 
through  a  commitment  to  demonstrate  excellence  in  their  learning  achievements.  As  one 
senior  administrator  stated,  "I  love  the  accountability  [associated  with  AISI]  because  it 
gives  me  the  leverage  to  go  to  schools  and  say,  'We  are  accountable  for  these  dollars.'" 

This  discourse  of  accountability  both  supported  and  was  supported  by  a  strong  cultural 
belief  that  the  primary  work  of  learning  was  to  ensure  that  teachers  had  access  to  the  very 
best  research  and  practices  available,  which  would  enable  the  ongoing  improvement  of 
student  learning  in  schools.  While  there  was  evidence  of  considerable  collaboration 
among  teachers  in  local  school  sites,  its  principal  purpose  was  not  to  create  knowledge 
but  to  enable  efficient  distribution  and  dissemination  of  "best  ideas  and  practices."  As 
one  teacher  put  it:  "I  would  leave  the  [sorting  through  all  the  research  in  this  area]  to  the 
consultants.  Let  them  figure  out  what  is  going  on  and  then  share  it  with  us." 

3.3  How  do  these  conceptions  of  learning  and  learners  contribute  to  enacted  roles  and 
relationships  in  Hearthstone? 

In  Hearthstone,  people  and  systems  are  organized  in  terms  of  how  they  might  best 
provide  services  to  one  another  and  to  the  local  needs  of  schools  and  communities,  based 
on  the  values  and  commitments  established  by  the  District  community  as  a  whole.  People 
were  well  aware  of  the  responsibilities  of  their  roles.  They  understood  they  were  assigned 
these  roles  because  of  their  specific  expertise,  so  they  could  provide  optimal  service  to 
others. 

There  is  good  connectivity  within  schools  among  teachers,  administrators,  and  the 
community,  with  a  well-developed  understanding  of  how  research  and  best  practices 
enable  the  development  of  success  in  learning  for  students.  However,  teachers  do  not 
have  a  global  sense  of  what  is  happening  in  the  District.  'Weak  links'  that  support 
collaboration  between  and  among  different  schools  in  the  District  were  absent,  largely 
because  attention  and  value  is  placed  on  being  very  efficient  with  knowledge  gathering 
and  dissemination,  not  knowledge  production. 

Most  teachers  were  not  sure  how  decisions  were  made  about  AISI  projects  and  were  not 
clear  about  what  was  happening  across  the  District.  When  asked  directly  about  how 
decision-making  relative  to  AISI  occurred,  one  teacher  stated,  "I  don't  know  anything 
about  that."  Another  teacher  explained,  "I  appreciate  there  are  people  who  have  a  vision 
in  certain  areas  and  have  taken  the  time  to  invest  because  I  know  it's  a  big  job.  I  haven't 
been  part  of  developing  projects,  but  I  know  others  have  been.  It's  a  huge  process." 

3.4  How  do  these  enacted  roles  and  relationships  contribute  to  the  developing  and 
maintaining  of  the  organizational  mindset  that  guides  Hearthstone? 
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Hearthstone's  system  of  trust  has  enabled  a  hybrid  "top-down/bottom-up"  approach  to 
decision  making.  One  senior  administrator  explained,  "It's  a  focused  goal-development 
process  with  a  lot  of  communication  and  a  lot  of  feedback."  This  development  process  is 
led  by  a  steering  committee  that  sets  the  overall  direction  for  A  IS  I  projects  followed  up 
by  a  review  committee  that  recommends  specific  projects  that  are  eventually  ranked,  with 
some  developed  into  proposals.  People  occupying  all  roles  are  involved  in  these 
processes;  however,  primary  decision-making  authority  rests  with  the  senior  District 
leadership. 

We  received  mixed  reviews  and  messages  about  how  these  processes  are  experienced. 
Most  were  supportive.  One  teacher  explained,  "We  buy  into  [the  AISI  projects]  quickly 
because  we  trust  the  people  that  are  offering  them  to  us."  Another  commented,  "I  trust 
[names  senior  District  administrator]  and  [names  AISI  coordinator]  have  gone  through 
and  worked  the  profiles  for  the  AISI  funds,  so  I  know  that  they  have  done  their  jobs." 
The  layer  of  specialist  consultants  in  the  District  office  or  in  schools  select  appropriate 
ideas  and  practices  and  ensure  they  are  known  and  used  in  schools.  The  consultants  are 
attuned  to  what  is  happening  across  the  District.  They  are  the  channels  of  knowledge 
exchange  between  and  among  research/practice,  schools/central  administration.  The 
schools,  then,  take  up  these  ideas  and  integrate  them  into  their  local  priorities  and  needs. 

3.5  How  does  this  organizational  mindset  influence  the  ways  AISI  is  understood, 
conceptualized  and  operationalized  within  Hearthstone? 

In  the  Hearthstone  District,  a  robust  service  layer  was  developed  and  expanded,  in  part 
through  AISI  funding,  to  provide  a  well-developed  interface  with  relevant  educational 
research.  One  teacher  commented,  "We  just  know  when  it's  coming  down  the  pipe  that 
it's  tried-and-true,  based  on  good  research,  so  we  know  that  it  will  be  a  successful 
project,  and  we're  wilUng  to  put  our  efforts  into  it." 

One  consultant  explained,  "AISI  has  helped  us  to  create  new  networks  of  support  that 
were  not  there  before.  We're  now  a  whole  new  organism."  A  specialist  teacher 
commented,  "Whether  we  are  lead  teachers  or  consultants,  the  amazing  opportunity  for 
professional  development  was  embedded  into  our  practices  at  every  level.  And  all 
informed  by  new  research  on  best  practices."  It  is  these  "best  practices"  that  have  created 
the  connectivity  in  this  District.  Although  teachers  across  schools  did  not  seem  to  be 
aware  of  one  another's  specific  communities  or  projects,  they  were  aligned  around 
similar  ideas  and  practices,  emerging  from  their  connections  to  centralized  specialist 
consultants. 

With  considerable  vision  and  leadership  from  the  Associate  Superintendent  and  AISI 
Coordinator,  the  District  has  created  a  strong  layer  of  trusted  specialist  consultancy, 
which  has  become  important  connective  tissue  among  university-based  research,  central 
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administration's  goals  and  priorities,  and  local  needs  of  schools  in  the  District.  One 
teacher  summarized  it  thus:  "We  are  not  a  chain  of  command,  but  a  chain  of  trust.  It 
seems  to  work  really  well." 

3.6  Summary 

In  the  mid-1990s,  Hearthstone  was  quite  distributed,  lacking  systemic  focus.  The 
District's  central  administration  had  begun  to  address  this  issue  a  few  years  prior  to 
AISFs  introduction,  implementing  a  District-wide,  research-focused  professional 
development  project  for  all  teachers.  This  initiative  was  commonly  described  as  a 
defining  moment  in  the  District's  unfolding  narrative,  being  emblematic  of  a  strong 
commitment  to  service  and  support  for  all  and  a  way  of  achieving  systemic  coherence. 

The  centralized  approach  to  that  initiative  set  the  stage  for  Cycle  1  of  AISI,  which  was 
developed  around  several  District-wide  projects  that  were  highly  centralized  and  oriented 
toward  teacher  support.  These  projects  brought  forth  new  forms  of  collaboration  and 
connection  not  previously  possible  in  the  District.  As  well,  in  conjunction  with  projects 
initiated  in  Cycles  2  and  3  Hearthstone  now  had  the  opportunity  to  create  a  robust 
specialist/consultancy  service  layer.  This  has  helped  to  amplify  the  District's  narrative  of 
service  and  support. 

However,  while  this  new  service  layer  is  effective,  it  contributes  to  a  culture  of 
knowledge-distribution  and  not  also  knowledge-production.  It  operates  well  in  moving 
new  ideas  from  a  central  authority  to  individual  teachers,  supporting  the  emergence  of 
redundancy  through  a  "common  language"  and  being  "on  the  same  page".  However,  the 
system  seems  unable  to  recognize,  much  less  capitalize  on,  the  tremendous  diversities  of 
expertise  and  interest  present  within  the  District  community. 

Hearthstone  has  evolved  over  the  last  15  years  from  being  quite  distributed  to  much  more 
centralized.  At  the  moment,  its  AISI  projects  operate  within  a  somQ^whsLt  fragmented 
network.  This  evolution  has  been  greatly  enabled  and  enhanced  by  AISI,  perhaps  even 
triggered  by  it.  The  service  layer  that  has  arisen  through  AISI  projects  has  become  an 
effective  clearinghouse  for  ideas  and  practices.  At  the  same  time,  some  schools  (or 
nodes)  do  not  feel  as  strongly  connected  to  other  nodes  as  they  might,  owing  in  part  to  the 
fact  that  not  all  schools  are  involved. 

4.  Arrowhead  School  District:  The  work  of  the  district  is  management. 

4. 1  How  does  Arrowhead  conceptualize  its  work? 

A  strong  work  ethic  organizes  the  overall  mindset  of  Arrowhead.  This  was  evident  in  the 
orienting  meeting  with  central  and  school  administrators,  teachers.  School  Board 
members,  consultants  and  parents  where  there  were  many  references  to  creating  and 
supporting  structures  that  would  ensure  goal-oriented  efficiency  and  accountability.  The 
Superintendent  stated,  "There  has  to  be  a  grand  scheme.  We  set  the  directions  and  then 
schools  build  their  plans." 
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There  is  a  strong  commitment  to  discerning  "best  practices"  emerging  from  educational 
and  other  research  on  learning  that  might  help  to  improve  student  learning.  Arrowhead 
has  encouraged  school-based  AISI  projects,  which  in  some  schools  have  become 
embraced  by  teachers  over  time.  The  AISI  coordinator  in  one  school  explained,  "We  tried 
to  make  sure  that  everybody  got  a  little  bit  of  AISI;  some  people  got  more,  but  everybody 
could  participate." 

Paradoxically,  Arrowhead's  explicit  work  ethic  has  created  some  loss  of  connectivity 
among  schools  and  between  schools  and  the  District  office.  A  number  of  teachers  and 
administrators  were  very  concerned  that  the  decision  to  centralize  AISI  with  one  project 
was  not  only  curtailing  most  school-based  initiatives,  but  also  communicating  disregard 
for  systems  of  in-school  and  inter-school  connectivity  that  earlier  projects  had  created. 
The  emphasis  on  managerial  efficiency  by  consolidating  all  AISI  funds  into  one  major 
project  is  creating  a  networking  shift  in  the  District  -  from  being  more  distributed  to 
more  centralized.  As  one  teacher  put  it,  "So  we've  moved  from  individualism  to  a  more 
collective  whole  -  where  now  a  [steering]  committee  sets  the  expectations  and  how  we 
are  to  meet  them." 

4.2  How  does  this  conception  of  work  influence  Arrowhead's  implicit  conceptions  of 
learning  and  learners? 

In  the  Arrowhead  District,  learning  is  understood  as  the  work  of  the  individual.  These 
efforts  of  individuals  should  have  discernible  and  measurable  products.  While  some 
culture  of  collaboration  had  been  established  in  the  two  schools  we  visited,  collaboration 
did  not  extend  much  outside  school  subject  area  departments,  among  grade  levels,  or 
across  schools.  As  one  administrator  stated,  "We're  trying  to  connect  to  other  schools, 
but  we're  not  having  much  luck." 

Most  people  with  whom  we  spoke  discussed  learning  as  the  process  of  developing 
competence  and  producing  artifacts  demonstrating  that  competence.  One  teacher 
commented,  "AISI  has  provided  a  lot  of  good  materials,  teacher  resources,  books,  and 
student  books.  [We  appreciate  these  because]  we're  so  busy  trying  to  survive."  One 
school  had  become  very  innovative  by  archiving  teacher-generated  curriculum  resources 
online  with  access  via  interactive  whiteboards  located  in  each  classroom.  However,  one 
teacher  was  concerned  that  teachers  did  not  seem  to  have  opportunities  to  collaboratively 
create  or  share  curriculum  resources.  Collaboration  often  entailed  more  work  with  little 
benefit.  She  commented,  "[The  District]  does  not  provide  funding  for  us  to  do  the  prep 
work  for  subs  while  we're  sharing  our  work  with  other  teachers.  Sometimes  I  feel  like 
I'm  the  only  one  sharing." 

Overall,  effective  learning  and  school  change  were  seen  to  occur  in  an  ordered  and 
predictable  environment,  where  the  direction  and  tone  for  these  changes  were  set  by 
senior  administration.  One  school  principal  stated,  "AISI  has  helped  me  to  move  the 
school  forward  to  where  I  wanted  it  to  go."  In  this  learning  orientation,  failure  is  to  be 
avoided  if  possible  and,  when  noticed,  is  a  problem  to  be  solved.  As  one  administrator 
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explained,  "There  are  four  major  projects  in  the  District  of  varying  quality.  ...  Why  waste 
the  money  and  keep  on  going?" 

Architectural  and  monetary/value-exchange  metaphors  were  often  used  to  describe  both 
learners  and  structures/processes  of  learning.  For  instance,  one  senior  administrator 
explained,  "The  question  becomes  where  one  puts  the  money  so  we  can  get  the  best  value 
for  our  dollar."  These  discourses  of  efficiency  characterized  comments  about  how  AISI 
funding  had  been  used  very  efficiently  and  responsibly  to  create  effective  learning 
environments  for  students.  While  the  schools  we  visited  seem  to  have  found  ways  to 
cohere  internally  into  a  collective  mindset  around  learning,  there  was  little  evidence  that 
teachers  had  a  more  global  understanding  of  how  other  schools  in  the  District  are 
organized. 

4.3  How  do  these  conceptions  of  learning  and  learners  contribute  to  enacted  roles  and 
relationships  in  Arrowhead? 

Arrowhead  seems  to  hold  three  interrelated  conceptions  or  beliefs:  learners  are  oriented 
toward  the  mastery  of  established  content;  learning  is  the  work  of  the  individual;  and  hard 
work  and  efficiency  are  greatly  valued.  These  connect  to  a  clearly  articulated  hierarchy  of 
authority  and  centralized  decision-making.  Senior  administrators  take  seriously  their 
responsibility  for  working  closely  with  the  Board  to  use  funds  appropriately  (in 
accordance  with  Alberta  Education  priorities)  and  to  set  directions  for  innovations 
deemed  necessary  for  the  entire  District.  There  are  strong  collegial  relationships  among 
senior  District  administrators  and  elected  Board  members,  with  opportunities  for  shared 
decision-making  at  this  top  level. 

Within  this  organizational  structure,  individual  teachers  are  primarily  responsible  for 
planning  and  developing  their  own  professional  learning,  which  is  evidenced  and 
measured  in  the  production  of  physical  artifacts  that  are  not  unlike  those  in  place  to 
monitor  and  "authentically  assess"  the  learning  of  their  students.  The  principal's  role 
appeared  to  focus  on  effective  management  of  teachers'  activities  by  ensuring  that  some 
resources  and  release  time  are  provided  for  teachers  to  pursue  their  individual 
professional  learning  goals.  This  structured  approach  does  not  appear  to  set  a  context  for 
collective  knowledge  production  or  dissemination,  requiring  little  collaboration  and/or 
communication  between  and  among  teachers  or  administrators.  One  teacher  commented: 
"All  of  the  departments  seem  to  have  trouble  communicating  with  one  another."  When 
collaboration  occurs,  it  is  primarily  to  distribute  ideas  and  workloads  in  an  equitable  and 
efficient  manner. 

4.4  How  do  these  enacted  roles  and  relationships  contribute  to  the  developing  and 
maintaining  of  the  organizational  mindset  that  guides  Arrowhead? 

A  commitment  to  efficiency  also  seems  to  have  oriented  Arrowhead  to  focus  attention  on 
one  major  project.  This  is  seen  as  a  way  to  maximize  the  impact  of  AISI  funding  on 
teacher  development  and  student  learning.  As  the  Superintendent  explained,  "A  lot  of 
money  goes  to  professional  development.  Most  of  it  is  sub  costs,  conference  costs  and 
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travel  costs.  If  [teachers]  are  going  to  one  conference,  it  may  be  15  people  going  to  it  - 
and  that's  a  huge  amount  of  money.  You  could  bring  in  that  person  for  several  days  and 
everybody  in  the  system  really  gets  to  see  them,  hear  them  and  work  with  them."  One 
major  challenge  for  this  District  seemed  to  be  getting  teachers  to  "buy  in"  to  centrally 
directed  AISI  projects,  mostly  because  the  move  to  centralization  meant  removing 
resources  that  had  once  been  locally  situated. 

One  of  the  effects  of  a  centralized  approach  to  management  and  decision-making  is  an 
overall  lack  of  information  within  the  system  of  how  AISI  monies  have  been  or  could  be 
used  to  improve  student  learning.  In  one  school,  where  funds  had  previously  been  used 
for  teacher  release  time,  one  teacher  commented,  "So  you  have  an  extra  prep,  and  now 
you're  sitting  in  your  room  wondering,  'How  is  this  going  to  help  me  to  do  anything?'" 
School  based  administrators  described  themselves  as  "mostly  cheerleaders."  One  teacher 
commented,  "I  have  a  feeling  that  some  of  our  administrators  might  not  know  what  AISI 
is  about." 

4.5  How  does  this  organizational  mindset  influence  the  ways  AISI  is  understood, 
conceptualized,  and  operationalized  within  Arrowhead? 

The  Board  of  Trustees  in  Arrowhead  works  very  closely  with  the  Superintendent  and  the 
AISI  Coordinator  to  determine  the  best  (most  fair,  efficient  and  effective)  uses  for  this 
funding,  relying  heavily  on  expressed  District  goals  and  values.  These  goals  and  values 
are  explicitly  tied  to  Alberta  Education  accountability  pillars.  This  past  year,  the  decision 
was  made  to  centralize  all  AISI  monies  into  one  initiative  that  would  involve  all  schools 
and  teachers  in  a  research-based  project  to  improve  instruction  at  all  levels.  The  AISI 
Coordinator  will  work  with  all  schools  in  the  District  to  organize  release  time  for  teachers 
who  are  participating,  and  also  to  provide  other  necessary  supports  to  ensure  that  these 
new  ideas  are  integrated  into  classroom  practices.  In  effect,  AISI  monies  are  being 
leveraged  with  other  professional  development  funds  in  the  District  to  create  a  more 
robust  approach  to  teacher  learning.  As  the  AISI  Coordinator  explained,  "Professional 
development  and  AISI:  They  are  not  different.  . . .  With  Cycle  4  I  am  using  funding  for 
teachers  to  collaborate,  so  that  they  can  participate  in  the  [name  of  the  initiative  that  was 
chosen]." 

It  is  not  yet  clear  whether  this  decision  will  create  greater  connectivity  in  this  District,  or 
whether  it  will  create  the  sort  of  alienation  teachers  sometimes  experience  when  a 
particular  approach  to  professional  development  is  mandated.  Teachers  with  whom  we 
spoke  expressed  concern  about  the  movement  to  centralize,  viewing  this  as  an  intrusion 
on  their  professional  development  autonomy.  Others  had  pragmatic  concerns,  related  to 
the  work  it  would  take  for  them  to  attend  the  required  days  of  professional  development 
mandated  by  the  District:  "Every  teacher  will  spend  five  days  at  workshops  [given  by  this 
person]  and  that  means  five  days  of  preparing  for  subs!"  While  supportive  of  the 
District's  initiative,  one  school  administrator  expressed  some  concern:  "With  Cycle  4, 1 
am  losing  the  funding  for  teachers  to  collaborate  in  the  school  and  that's  hard." 
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4.6  Summary 

Data  collected  from  the  Arrowhead  District  pointed  to  a  system  that  had  become  quite 
distributed.  There  was  evidence  of  very  good  quality  communication  in  nodes  of  the 
system  (including  within  each  school),  but  weak  communication  between  and  among 
nodes  and  across  administrative  levels.  \n  part,  this  has  emerged  from  a  long  history  of 
very  good  management,  where  central  administrators  have  excelled  at  using  public  funds 
well  and  have  responded  appropriately  to  mandates  of  Alberta  Education.  Indeed, 
references  to  this  strong  managerial  ethic  constituted  the  principal  site  of  coherence  in 
narratives  of  the  District  offered  by  its  members. 

Consistent  with  these  long-standing  narratives,  AISI  funds  have  been  used  to  amplify 
sound  management  structures,  focusing  on  projects  and  processes  that  are  clearly 
articulated  and  carefully  aligned  with  Alberta  Education  goals,  particularly  as  those  goals 
relate  to  school  improvement,  student  success,  and  district  accountability. 

Strong  management  has  also  contributed  to  pockets  of  excellence,  with  regard  to  a  few 
school-specific  AISI  projects.  These  undertakings  are  diverse,  powerful,  and  frequently 
referenced  as  exemplars  of  the  provincial  initiative.  At  the  same  time,  they  are  very 
localized.  There  is  little  sense  of  District-level  accomplishment  around  AISI.  In 
particular,  this  system  appears  to  lack  the  sort  of  communicative  redundancy  that  is 
necessary  to  either  an  effective  knowledge-disseminating  system  or  a  robust  knowledge- 
producing  system. 

It  is  important  to  note  that  in  Cycle  2  of  AISI,  the  Arrowhead  District  devoted  a 
substantial  portion  of  new  resources  to  the  creation  and  support  of  in-school  professional 
learning  communities  -  including,  in  particular,  considerable  release  time  for  teachers. 
However,  this  was  done  in  a  very  distributed  manner,  in  which  communications  across 
schools  and  levels  of  organization  were  not  well  developed.  Consequently,  the  intended 
"professional  learning  communities"  did  not  have  the  sort  of  system-wide  effect  that  was 
hoped.  They  did  operate  within  schools  to  good  effect,  but  beyond  school  walls, 
discourses  of  disconnection,  territoriality,  and  competition  prevailed.  As  a  result,  with 
respect  to  AISI,  the  District  seems  to  have  maintained  2i  fragmented  network  of  projects 
through  Cycles  2  and  3.  This  detail  has  not  gone  unnoticed,  and  was  in  fact  identified  by 
District  administrations  as  the  main  impetus  for  the  move  to  centralize  AISI  work 
through  a  single  major  project  in  Cycle  4. 

5.  Revisiting  the  Research  Questions 

5.7  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI?) 

The  Alberta  Initiative  for  School  Improvement  is  organized  around  an  explicit  awareness 
that  the  improvement  of  student  learning  is  intimately  entwined  with  improvements  to  the 
contexts  of  learning.  This  core  value  of  AISI  is  the  key  to  understanding  its  emergent 
values  of  collaboration  and  connection. 
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In  each  of  the  three  districts  we  visited,  across  all  three  of  the  AISI  cycles  to  date,  there 
has  been  a  dramatic  increase  in  opportunities  for  teachers  to  collaborate  in  shared 
learning  projects.  Teachers  have  chances  to  meet  with,  learn  from,  and  inform  one 
another.  This  has  been  of  immeasurable  value  -  with  regard  to  innovations,  educators' 
attitudes  toward  their  role,  and  appreciation  of  the  range  and  depth  of  teacher  expertise  in 
the  province. 

The  requirement  to  infuse  current  research  into  AlSI-related  projects  has  sparked 
considerable  connectivity,  as  each  district  developed  strategies  and  structures  to  gather 
and  infuse  new  thinking  into  its  system.  In  our  extensive  experience  with  school 
jurisdictions  across  the  country,  we  have  never  encountered  districts  with  greater 
awareness  of  or  tighter  links  to  university-based  research  and  researchers. 

5.2  Why  couldn't  jurisdictions  do  this  anyway  (without  AISI)? 

A  school  district  is  more  than  an  administrative  structure.  Each  of  the  three  districts  has  a 
core  narrative  that  is  coherent  and  stable,  that  is  rooted  in  history  and  anchored  to 
community,  and  that  informs  discussions  and  orients  decisions.  While  these  qualities  are 
vital  for  effective  and  efficient  day-to-day  operation,  they  can  be  limiting.  To  that  end, 
AISI  has  helped  to  interrupt  and  to  amplify. 

Like  all  complex  systems,  all  of  the  districts  we  studied  are  engaged  in  ongoing 
adaptation  as  new  educational  and  contextual  challenges  arise,  and  these  adaptive 
activities  certainly  preceded  the  introduction  of  AISI.  However,  through  challenging 
districts  to  innovate,  demanding  accountability,  and  infusing  a  level  of  uncertainty  around 
the  maintenance  of  funding,  AISI  is  providing  a  different-from-usual  source  of 
disequilibrium.  It  is  not  allowing  districts  to  slip  into  a  'comfort  zone'  or  to  do  'business 
as  usual.'  It  is  unlikely  that  this  sort  of  interruption  would  occur  without  AISI. 

AISI  has  also  amplified  the  work  of  each  district.  This  has  highlighted  important 
strengths  and  also  unearthed  some  less  positive  aspects.  Through  compelling  districts  to 
confront  such  matters,  AISI  is  presenting  an  occasion  for  systemic  transformation  that 
would  likely  not  otherwise  be  possible. 

5.3  Would  the  values  of  AISI  continue  without  funding? 

Our  strategy  for  addressing  this  question  was  to  attend  to  the  mindset  of  each  of  district. 
We  wondered  if  the  emergent  values  of  AISI  -  in  particular,  the  elements  of 
collaboration,  connection,  interruption,  and  ampUfication  -  were  consistent  with  the 
historicized  characters  of  each  district. 

In  one  case  (Pathways),  in  which  the  District  organized  its  work  around  a  narrative  of 
learning  and  learners,  it  is  likely  that  the  values  of  AISI  would  continue  without  direct 
funding,  given  the  District's  pre- AISI  history.  Collaboration  and  connectivity  are  deeply 
inscribed  in  the  District's  self-narrative,  and  would  likely  persist  even  without  targeted 
funding. 
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However,  with  their  work  organized  around  narratives  of  service  and  management, 
administrators  in  the  other  two  districts  acknowledged  that  differentiated  funding  was 
crucial  to  the  maintenance  of  AlSI-like  projects.  Some  expressed  anxiety  over  the 
possibility  that  AISI  funding  might  be  folded  into  base  budgeting,  let  alone  the  possibility 
that  it  might  be  discontinued  entirely.  In  districts  such  as  these,  without  clearly  marked 
funding,  the  sorts  of  projects  associated  with  AISI  would  fade  away. 

5.4  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

There  is  compelling  evidence  that  AISI  has  affected  subcultures  of  education  in  the 
province  (i.e.,  school  districts).  For  example,  above-noted  shifts  in  vocabulary  are  strong 
indicators  of  transformation.  Unfortunately,  it  is  impossible  to  attribute  responsibility  for 
such  changes. 

To  explain,  each  district  demonstrated  itself  to  be  a  responsive  and  adaptable  entity  -  a 
learner  -  built  on  coalitions  and  networks  that  operate  in  common  purpose  to  create 
coherent,  self-maintaining  systems.  However,  learners  must  also  be  resilient.  To  maintain 
coherence  they  must  resist  some  sorts  of  change  even  as  they  adapt  to  evolving 
circumstances.  This  point  is  especially  clear  around  district  mindsets  and  the  associated 
attitudes  toward  knowledge.  One  system  (Pathways)  manifests  the  power  and 
possibilities  of  a  knowledge-production  orientation;  the  other  two  seem  to  embody 
attitudes  that  limit  their  AlSI-related  projects  to  knowledge  dissemination.  This  key 
difference  might  serve  as  an  explicit  target  of  the  Initiative.  In  particular,  our  analysis 
suggests  two  important  points  of  emphasis:  resourcing  emergent  networks  and 
embodying  a  learning  mindset. 

Regarding  the  first  point  -  that  is,  resourcing  emergent  networks  -  it  bears  emphasizing 
that  none  of  the  people  we  met  were  able  to  point  to  either  horizontal  (between  and 
among  districts)  or  vertical  (across  levels  of  organization,  from  schools  through  the 
Ministry)  effects.  No  one  was  aware  of  what  was  going  on  in  other  districts  in  any  great 
detail.  What  little  was  known  appeared  to  be  accidental.  Given  the  pockets  of  remarkable 
innovation  in  the  province  and  deep  commonalities  in  interest,  expertise,  and  activity 
across  jurisdictions,  the  time  seems  right  for  some  interdistrict  networking  activity  that 
extends  beyond  the  short-term  encounters  of  annual  conferences  or  the  non-interactive 
structures  of  web-based  archives. 

As  for  the  second  point  -  that  is,  of  enacting  and  projecting  a  mindset  of  learning  or 
knowledge-production  -  we  return  to  the  example  of  the  Pathways  District.  Tellingly, 
even  within  Pathways,  Alberta  Education  is  not  perceived  as  part  of  the  learning 
community.  Rather  it  is  generally  seen  as  a  top-down,  disconnected,  and  dissemination- 
oriented  administrative  structure  -  that  is,  structurally  akin  to  the  Arrowhead  District. 
AISI  has  done  little  to  interrupt  this  perception.  Noting  that  culture  cannot  be  borrowed 
or  imposed,  but  arises  organically  in  the  dailyness  of  communication  and  shared  work, 
we  would  thus  end  our  report  with  the  recommendation  that  efforts  be  made  to  enact 
through  AISI  a  more  participatory  and  decentralized  relationship  with  districts  in  the 
province. 


50 


The  Learning  Mosaic 


Chapter  4:  AISI:  A  Qualitative  Case  Study 


By  Dennis  Shirley  and  Lori  McEwen 


1.  Introduction 

This  chapter  describes  a  quahtative  condensed  case  study  of  12  school  districts  in  the 
Alberta  Initiative  for  School  Improvement  (AISI).  This  study  employed  interviews  and 
focus  group  discussions  to  gather  data  about  the  meaning  and  value  of  AISI  as 
experienced  by  those  educators  and  district  personnel  who  are  most  involved  in  and 
responsible  for  issues  of  conceptualization,  implementation,  and  assessment  of  AISI 
activities  at  the  district  level.  The  study  was  solicited  by  AISI  after  a  colloquium 
convened  in  October  2008. 

For  this  study  12  districts  were  selected  by  Alberta  Education  to  be  representative  of  the 
diversity  of  public  school  authorities  in  Alberta  by  type,  size,  and  location.  Public 
districts  included  public,  Catholic,  division,  and  charter  school  authorities.  They  ranged 
in  size  from  just  1  school  with  under  300  students  to  over  200  schools  with  over  100,000 
students.  Geographically,  the  12  districts  were  spread  across  the  northern,  southern,  and 
central  zones  of  the  province;  they  spanned  metropolitan,  small  city,  town,  and  rural 
areas. 

To  explore  the  research  topic,  5  broad  research  questions  and  1 1  subsidiary  questions 
were  finalized  in  February  and  March  2009  through  a  process  of  consensus  by  the 
research  team  and  Alberta  Education.  The  5  broad  questions  were: 

1.  What  is  the  distinctive  theory-in-action  (change  architecture)  of  AISI? 

2.  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI?) 

3.  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

4.  Would  the  values  of  AISI  continue  without  funding? 

5.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 


These  were  supplemented  by  1 1  subsidiary  questions: 

1.  What  have  been  the  successes  of  AISI,  as  assessed  from  multiple  perspectives? 

2.  What  are  both  the  obvious  and  the  subtle  impacts  of  AISI? 

3.  How  is  AISI  lived  and  practiced  by  educators? 

4.  What  are  the  change  processes  at  play  for  administrators  and  teachers? 

5.  What  have  been  the  difficulties  and  challenges  of  AISI? 

6.  Has  AISI  encouraged  school  authorities  to  try  new  things? 

7.  Has  AISI  encouraged  those  involved  to  take  risks  and  to  be  more  innovative? 

8.  What  are  the  opportunities  to  expand  the  measures  of  AISI  projects  from  the 
vantage  point  of  complexity  theory? 
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9.  What  are  the  opportunities  to  disseminate  knowledge  generated  by  AISI  by  using 
its  networks  and  complexity  thinking?  Have  these  opportunities  been  used  to 
promote  change  across  AISI  jurisdictions? 

10.  How  has  AISI  influenced  policy  developments  at  the  school,  jurisdictional,  and 
provincial  levels? 

11.  What  are  the  implications  of  the  research  findings  for  AISI  as  a  work  in 
progress? 

The  questions  allowed  the  informants  to  explore  multiple  dimensions  of  their  work  with 
AISI  to  share  the  meaning  and  value  of  their  activities.  Informants  discussed  district  foci 
prior  to  their  involvement  in  AISI,  the  manner  in  which  AISI  enabled  them  both  to 
deepen  and  to  amplify  their  work,  and  the  value  of  AISI  as  a  change  strategy  that 
strengthened  their  communicative  connectivity  within  and  across  districts.  They 
elaborated  on  the  complexity  of  change  processes  and  described  the  manner  in  which 
district-level  and  school-based  leadership  evolved  to  support  AISI  activities.  Through 
these  interviews  a  rich  and  multifaceted  set  of  data  emerged  that  enabled  the  research 
team  to  discern  trends  and  patterns  in  AISI,  an  extraordinarily  ambitious  and  intentional 
change  initiative  in  Canada's  highest- achieving  province. 

This  summary  of  the  research  team's  findings  is  organized  to  focus  on  the  5  overarching 
questions  with  responses  and  commentaries  relevant  to  the  1 1  subsidiary  questions 
embedded  throughout  the  summary  report  for  purposes  of  further  clarification. 

1.1  Methodology 

The  research  team  sought  to  achieve  consistency  across  the  12  case  study  district  sites  by 
assuring  that  in  each  instance  individuals  in  similar  positions  were  interviewed.  These 
included: 

•  The  district  superintendent; 

•  Project  coordinators  and  affiliated  staff; 

•  District  school  improvement,  research  and  administrative  staff; 

•  School  principals  and  staff  involved  directly  in  project  implementation; 

•  Parent  and  community  participants  in  project  activities. 

Interviews  were  conducted  in  person  during  district  site  visits  and  were  semi-structured 
around  the  questions  with  subsidiary  probes  to  allow  for  maximum  rapport.  All 
interviews  were  transcribed  verbatim  and  subsequently  coded  to  identify  and  illuminate 
patterns  and  variations  by  drawing  comparisons  across  the  12  case  studies.  In  case  study 
research,  the  goal  is  to  generalize  to  theoretical  propositions.  Thus,  the  aim  was  to  map 
the  different  forms  of  project  and  district  learning  in  a  number  of  settings  and  contexts. 

The  research  team  also  visited  schools  and  classrooms,  which  was  especially  helpful  in 
terms  of  our  ability  to  see  AISI  themes  and  strategies  in  action.  Teachers  provided  us 
with  opportunities  to  observe  their  instruction  and  to  see  how  children  responded  to 
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pedagogical  and  curricular  changes  that  they  had  implemented  as  part  of  AlSI-funded 
projects.  Several  of  the  districts  had  prepared  samples  of  student  work  so  that  the 
research  team  could  track  directly  how  AISI  projects  were  influencing  student  learning 
over  time.  Districts  likewise  provided  the  research  team  with  copies  of  their  AISI  project 
proposals,  implementation  plans,  and  annual  reports.  Some  districts  contacted  parents  in 
advance  so  that  we  could  interview  them  about  AISI  activities  and  their  impacts  on  their 
children. 

1.2  The  Case  Study  Districts:  An  Overview 

Twelve  districts  were  selected  by  the  School  Improvement  Branch  of  Alberta  Education 
to  be  representative  of  public  school  authorities  for  this  qualitative  case  study.  The  12 
districts  provided  an  extraordinary  breadth  and  range  of  AISI  projects  in  diverse 
geographical  settings  with  their  own  local  histories,  cultures,  and  signature  practices. 
Some  themes,  such  as  improving  formative  assessment  practices,  recurred  repeatedly  in 
the  12  districts.  Yet  more  important  than  the  adoption  of  any  single  consistent  approach, 
AISI  across  the  districts  was  seen  as  a  crucial  ally  in  assisting  educators  to  probe  deeply 
into  teaching  and  learning  with  students'  very  best  interests  in  mind. 


2.  The  Research  Findings 

2.1  What  is  the  distinctive  theory-in-action  (change  architecture)  of  AISI? 

AISI  is  a  program  that  encourages  creativity  and  innovation  and  supports  a  wide  range  of 
district-led  improvement  projects  throughout  Alberta.  AISFs  theory-in-action  empowers 
educators  to  develop  professional  and  intellectual  projects  based  on  their  own  locally- 
created  needs  assessments  and  subsequent  initiatives  for  self-initiated  change.  Over  the 
first  three  cycles,  more  than  1,700  AISI  projects  have  been  funded. 

A  stance  of  what  may  be  characterized  as  "active  trust"  from  Alberta  Education  towards 
teachers  and  school  leaders  has  produced  hundreds  of  locally-generated  initiatives  that 
have  catalyzed  educators  to  explore  new  routes  to  teaching  and  learning  that  often  are 
precluded  by  more  orthodox  school  reform  strategies.  AISI  projects  entail  a  tremendous 
variety  of  undertakings.  They  span  everything  from  projects  serving  language  minority 
students  to  new  instructional  approaches  that  develop  music  as  the  central  component  of 
an  elementary  school  curriculum  to  schools  endeavoring  to  improve  student  learning 
through  differentiated  instruction,  high-school  dropout  prevention,  or  technology 
infusion.  Some  projects  emphasize  parent  and  community  engagement;  specific 
academic  disciplines  such  as  literacy,  mathematics,  or  science;  or  transdisciplinary  topics 
that  allow  secondary  humanities  and  science  faculty  to  collaborate  on  local  ecologies  and 
their  histories,  for  example. 

In  addition  to  the  diversity  of  their  content  areas,  AISI  funding  supports  variegated 
district-level  and  school  structures.  Some  are  relatively  centralized  change  initiatives — in 
which  four  full-time  AISI  consultants  work  together  with  a  technology  expert  from  a 
district  office,  for  example — while  others  represent  more  diffused  models  of  change,  in 
which  large  numbers  of  teachers  dedicate  one-fifth  of  their  time  in  their  buildings  to  AISI 
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projects  while  directing  the  other  four-fifths  of  their  time  to  the  traditional  instructional 
tasks  of  a  classroom  teacher.  Most  projects  supported  PLCs  among  classroom  teachers, 
but  other  districts  have  gone  further  by  creating  PLCs  for  administrators  to  provide  them 
with  similar  structures  to  promote  organizational  development.  In  rural  communities, 
PLCs  can  stretch  across  schools  by  bringing  together  grade-level  teachers  to  share 
challenges  and  swap  potential  solutions  with  one  another  over  time.  AISI's  change 
architecture  allows  districts  and  schools  to  evolve  their  project  foci  and  structures,  with 
the  three-year  cycles  providing  regular  interludes  for  reviewing  previous  activities  and 
striking  out  in  new  directions. 

The  creative  and  bottom-up  possibilities  embedded  in  AISI's  change  architecture  explain 
the  universal  enthusiasm  for  AISI  of  project  participants.  One  charter  school  educator 
commented  that  "The  whole  structure  of  AISI  is  actually  great,  because  it  keeps  a  lot  of 
actual  freedom  in  how  you  are  going  to  work  on  your  project.  And  that  really  allows  for 
so  much  involvement,  individual  involvement  of  people,  they  feel  part  of  it.  Like  you're 
part  of  the  process."  Educators  referred  consistently  to  the  value  they  placed  on  AISI  as  a 
network  that  granted  them  freedom  to  address  all  of  the  complexities  of  their  everyday 
classroom  lives  while  also  providing  supports  to  develop  long-term  change  strategies  in 
their  buildings.  This  ability  to  participate  in  ground-level  theorizing  about  long-term 
goals,  and  to  acquire  the  professional  development  that  would  enable  educators  to  set 
structures  and  processes  in  place,  puts  AISI  at  the  forefront  of  contemporary  efforts  to 
overcome  educators'  "presentism"  or  short-term  thinking,  a  problem  long  recognized  as  a 
central  impediment  to  organizational  learning  by  sociologists  of  education. 

The  freedom  and  creativity  AISI  extends  to  its  participants  is  accompanied  by  a  strong 
accountability  component  that  requires  all  projects  to  complete  annual  reports  as  well  as 
more  detailed  analyses  at  the  end  of  each  three-year  cycle.  These  accountability 
components  require  educators  systematically  to  gather  data  about  the  nature  and  types  of 
interventions  and  project  outcomes  as  well  as  of  the  number  of  students  impacted  and  the 
number  of  staff  involved  in  the  projects.  Many  informants  complained  that  the  amount  of 
detail  required  by  the  reports  was  excessive,  but  most  also  understood  and  appreciated 
that  in  the  current  policy  climate  emphasizing  accountability,  educators  can  no  longer 
expect  to  receive  funding  without  evaluation  and  reflection  being  built  into  the  project 
design.  In  addition  to  the  project-level  reports  conducted  by  educators  in  districts  and 
schools,  rigorous  external  reports  conducted  by  university  faculty  document  AISI  impacts 
both  as  an  aggregate  as  well  as  for  special  groups  such  as  parents  and  First  Nations, 
Metis,  and  Inuit  students.  Educators  stated  that  these  reports,  printed  by  Alberta 
Education  in  reader-friendly,  well  designed  formats,  gave  them  "a  jumpstart  to  what  you 
want  to  do  ...  Instead  of  having  to  go  to  the  great  pool  of  research  that  is  out  there,  you 
actually  have  sort  of  a  head  start  into  what  is  working." 

In  addition  to  locally-generated  and  driven  school  improvement — characterized  by 
educators  as  "grassroots"  in  nature — AISI  has  fostered  networks  of  districts  coming 
together  through  cross-site  visits  and  regional  and  provincial  AISI  conferences.  These 
events  enable  educators  to  learn  from  each  other's  work  and  to  hear  outside  speakers  and 
consultants  who  inform  them  of  new  research  findings  that  are  relevant  to  their  everyday 
classroom  interactions  with  students,  colleagues,  and  parents  and  community  members. 
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Many  informants  spoke  of  the  power  and  importance  of  outside  consultants  with  solid 
research  records  as  well  as  practical  experience  in  real  schools  and  classrooms  in 
inspiring  their  thinking. 

Educators  asserted  that  AISFs  change  architecture  helped  them  to  learn  to  think  about 
themselves  as  researchers.  Through  AISI  project  leaders  and  consultants,  teachers 
learned  to  use  appropriate  tools  and  methodologies  to  improve  their  instruction.  One 
appreciative  teacher  stated,  "AISI  I  feel  has  helped  our  teachers — has  almost  given  them 
more  importance  because  they  are  part  of  the  research  team,  and  they've  had  to  learn 
about  researching  skills." 

Three  topics  emerged  in  the  course  of  the  interviews  and  focus  group  discussions  that 
AISI  may  wish  to  address  in  relationship  to  its  change  architecture.  First,  most 
informants  wanted  additional  help  with  effective  strategies  for  improving  parent  and 
community  engagement.  Second,  some  educators  indicated  that  they  were  overloaded 
with  new  projects  and  needed  assistance  with  reviewing  and  abandoning  outdated 
programs.  Finally,  more  explicit  attention  to  leadership  development  may  be  warranted. 

2.2  What  is  the  value  of  AISI?  {What  are  the  values  of  AISI?) 

Informants  agreed  that  AISI  is  catalyzing  authentic  and  deep  conversations  about 
teaching  and  learning  that  are  contributing  to  a  richer  repertoire  of  instructional  practices 
and  improved  student  achievement  in  Alberta.  They  credited  AISI  with  giving  them  new 
ways  to  observe  student  learning,  identify  obstacles  to  achievement,  and  revise 
instruction  so  that  their  students  learn  at  high  levels.  None  of  them  viewed  AISI  as  a 
distraction  (with  the  single  exception  of  what  they  viewed  as  excessively  onerous 
accountability  requirements).  The  educators  and  all  of  the  parents  interviewed  credited 
AISI  with  creating  grass-roots  level  excitement  about  teaching  and  learning,  and  with 
activating  educators  to  develop  their  own  needs  assessments  and  pilot  projects  to  address 
the  needs  of  their  own  schools  and  communities. 

These  achievements  of  AISI  are  especially  noteworthy  given  the  extensive  research 
literature  that  documents  the  ways  in  which  many  school  cultures  exacerbate  teachers' 
conservatism,  individualism,  and  short-term  thinking  or  "presentism."  (Cohn  & 
Kottkamp,  1993;  Little,  1990;  Lortie,  1975).  By  exposing  educators  to  alternative  sets  of 
practices  and  by  embedding  ongoing  support  into  schools  through  AlSI-funded  lead 
teachers  and  consultants,  AISI  has  helped  to  overcome  teachers'  conservatism  and  to  re- 
ignite  their  curiosity  about  new  and  better  ways  of  teaching  their  students.  The  many 
PLCs  and  AISI  teams  that  have  been  constituted  in  project  elementary  schools  seem  to 
have  replaced  individualism  (also  called  "privatism")  with  a  more  collective 
understanding  of  peer  learning  and  exchange,  although  many  secondary  schools  generally 
still  exhibit  faculty  identification  with  individual  subject  matter  expertise  more  than  a 
collaborative  approach  to  foundational  issues  of  teaching  and  learning. 

As  a  consequence  of  these  positive  attributes  that  combine  both  professional  expertise 
with  public  democracy,  educational  leaders  continually  stated  that  the  return  on 
investment  with  AISI  is  superb.  When  superintendents  were  asked  point-blank  whether 
they  would  prefer  unencumbered  funds  or  those  specifically  designated  for  AISI,  they  all 
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preferred  the  latter,  asserting  that  AISI  consultants,  values,  and  networking  opportunities 
possessed  high  value  for  their  jurisdictions.  Likewise,  not  one  classroom  teacher 
interviewed  perceived  AISI  as  an  unwanted  imposition  or  a  fad.  Whether  they  were 
superintendents,  principals,  or  teachers,  no  informants  stated  that  they  would  be  able  to 
make  better  use  of  AISI  funding  in  their  districts  if  it  were  given  to  them  with  no  strings 
attached. 

2.3  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

Many  educational  leaders  stated  that  they  had  numerous  previous  projects — related  to 
technology  use,  anti-bullying  initiatives,  character  education,  or  differentiated 
instruction — that  in  many  ways  manifested  the  same  core  humanistic  values  that  they 
identified  in  AISI.  They  viewed  these  values  as  oriented  around  a  child-centered 
philosophy  of  education  that  included  children's  subjective  well-being  as  well  as  their 
academic  success  as  part  of  the  core  mission  of  their  schools  and  districts.  To  this 
degree,  AISI  may  be  viewed  as  an  extension  and  amplification  of  principles  that  districts 
already  possessed.  Educators  tended  to  view  AISI  not  so  much  as  the  point  of  departure 
for  new  values,  but  rather  as  a  funding  source  that  enabled  them  to  realize  the  values  that 
they  had  always  had  but  often  became  diluted  in  the  press  of  events. 

When  questioned  about  AISI  and  funding,  one  superintendent  commented,  "We  could 
have  done  it  without  the  encouragement,  but  not  without  the  funding  . . .  we're  the  kind  of 
district  that  doesn't  need  to  be  pushed  to  do  things.  But  without  the  funding,  for  sure, 
more  difficult."  A  charter  school  principal  was  crystal  clear:  "Without  the  funding,  we 
couldn't  afford  it  ourselves." 

Districts  needed  funding  to  support  AISI  consultants,  to  provide  teachers  with  release 
time  to  learn  from  their  colleagues,  to  purchase  resources,  and  to  send  teachers  to 
professional  development  activities  such  as  the  annual  conferences  of  the  Alberta 
Assessment  Consortium.  In  remote  rural  districts,  the  opportunity  to  leave  one's  small 
town  and  to  learn  about  recent  research  findings  at  AISI  or  AAC  conferences  was 
especially  cherished.  AISI  funding  for  rural  districts  was  viewed  as  a  vital  lifeline  to  gain 
access  to  new  ideas  and  research  findings  in  education  and  to  establish  lateral  learning 
networks  with  other  educators  with  more  experience  in  the  practical  implementation  of 
the  findings. 

There  is  evidence  of  school  divisions  sustaining  projects  beyond  AISI  funding,  which 
tells  us  that  it  may  indeed  be  possible  that  the  jurisdictions  could  have  done  these 
innovative  projects  and  activities  without  AISI.  Some  districts  institutionalized  lead 
teacher,  project  leader,  and  consultant  positions  at  the  completion  of  a  given  AISI  cycle. 
AISI  provided  an  effective  career  ladder  for  many  teachers  by  giving  them  a  chance  to 
identify  a  special  area  of  expertise,  hone  it  in  collaboration  with  colleagues,  and 
eventually  acquire  the  confidence  and  skills  to  take  on  roles  as  AISI  project  leaders  in 
their  buildings  or  coordinators  or  consultants  in  their  districts.  Since  some  districts  began 
implementing  meso-level  policy  changes  such  as  assessing  principals  based  on  their 
efficacy  at  embedding  practices  from  AfL  in  their  schools,  AISI  actually  seems  to  have 
become  something  of  a  misnomer — not  so  much  sponsoring  initiatives  as  embedding  new 
norms  and  expectations. 
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2.4  Would  the  values  of  AISI  continue  without  funding? 

Many  participants  from  elementary  schools  said  that  the  cultures  of  their  schools  had 
changed  and  that  the  practices  that  came  about  due  to  AISI  were  now  embedded  in  their 
schools.  They  stated  that  while  AISI  was  indispensable  in  catalyzing  change,  AISI 
values  were  now  embedded  in  their  schools  and  would  continue  without  funding. 
Secondary  school  principals  and  teachers,  on  the  other  hand,  could  point  to  individual 
departments  or  clusters  of  faculty  that  had  changed,  but  also  acknowledged  the  presence 
of  reluctant  faculty  in  their  schools.  For  them,  AISI  values  still  required  funding  to 
enhance  a  larger  cultural  shift  towards  a  greater  focus  on  student  learning. 

Several  high  school  principals  stated  that  they  were  now  on  the  cusp  of  reculturing  their 
schools  to  help  teachers  to  discover  new  ways  to  engage  their  students  and  improve  high 
school  completion  rates.  These  principals  were  exploring  innovative  ways  of  teaming 
faculty  across  disciplines  and  embedding  supports  in  PLCs  that  help  faculty  to  overcome 
the  initial  obstacles  that  they  encounter  in  modifying  traditional  instructional,  curricular, 
and  assessment  practices  for  struggling  learners.  Research  (Hargreaves,  1994; 
McLaughlin  &  Talbot,  2001;  Muncey  &  McQuillan,  1995)  indicates  that  high  school 
faculty  generally  need  continual,  classroom-based  support  from  colleagues  and  outside 
consultants  to  shift  the  focus  of  their  attention  from  the  transmission  of  their  disciplinary 
expertise  to  their  students'  learning  styles.  For  AISI  values  to  continue  to  spread  at  the 
secondary  school  level,  then,  continued  funding  is  likely  to  be  necessary  for  its  projects  in 
high  schools,  especially  if  Alberta  is  to  succeed  in  improving  high  school  graduation 
rates. 

2.5  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

For  many  informants,  the  dominant  culture  of  education  in  Alberta  is  defined  by  student 
achievement  results  on  the  Provincial  Achievement  Tests  (PATs).  Educators  knew  that 
the  public  dimension  of  their  students'  achievement  results  reflected  on  them  and  that 
poor  years  resulted  in  defensiveness  and  loss  of  morale  and  good  years  built  confidence 
and  momentum.  They  knew  that  their  province  has  been  the  highest  achiever  in  Canada 
on  national  and  international  tests  for  many  years,  and  that  these  results  in  many  ways 
have  come  to  give  Alberta  a  unique  educational  identity  as  perceived  both  by  insiders  and 
outsiders.  Even  when  educators  disliked  the  PATs  and  were  articulate  about  their 
limitations,  they  still  defined  themselves  in  many  ways  by  them,  as  this  principal's 
remarks  convey:  "I  would  say  that  I'm  proud  of  the  fact  that  once  we  won  the  award  for 
the  fastest  improving  school  in  the  province,  and  two  or  three  other  times  we've  been 
nominated  to  be  in  the  top  15  schools  in  the  province  according  to  their  measure.  But 
it's  not  a  measure  that  I  agree  with  and  support." 

On  the  whole,  we  observed  a  familiar  phenomenon  in  regard  to  the  multiple  perspectives 
that  surfaced  in  our  interviews:  superintendents  were  by  and  large  the  most  enthusiastic 
about  the  PATs;  principals  were  critical,  but  not  overwhelmingly  so;  and  classroom 
teachers  considered  them  to  be  a  distortion  of  the  learning  process,  especially  at  the 
elementary  school  level. 
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Intentionally  or  not,  AISI's  change  architecture  has  led  educators  to  question  and  in  some 
instances  to  challenge  the  logic  of  Alberta's  PATs.  When  this  occurs,  AISI  is  indeed 
changing  the  culture  of  education  in  Alberta  by  shifting  educators'  focus  from  summative 
to  formative  assessments,  with  the  attendant  learning  opportunities  that  the  latter  implies. 
Educators  value  AISI  because  it  provides  them  with  the  sanction  and  financial  support  of 
Alberta  Education  to  raise  larger  questions  about  teaching  and  learning  that  go  beyond 
test-preparatory  activities— and  to  modify  their  practices  accordingly.  AfL  was  especially 
praised  by  educators  in  AISI  project  sites.  They  stated  that  AfL  has  helped  them  to  work 
more  closely  with  students  to  develop  criteria  of  excellence  in  their  work,  to  provide 
models  of  excellence,  and  to  persist  with  students  in  revising  their  work  until  all  students 
have  demonstrated  excellence. 

Educators  reported  great  "psychic  rewards" — a  key  feature  of  job  satisfaction  for  teachers 
in  their  teaching  after  adapting  new  strategies  that  allowed  all  of  their  students  to  achieve 
at  high  levels.  As  a  result  of  AfL  in  particular,  one  AISI  consultant  stated,  "Teachers  are 
engaging  in  a  variety  of  assessment  strategies  that  are  meeting  a  variety  of  needs  of 
students  ...  they're  really  triangulating  the  evidence  ...  making  observations,  listening  to 
conversations,  and  product  work  ...  it's  been  huge.  And  we're  going  in  such  a  great 
direction,  right  now,  that  it's  really  exciting." 

AISI  educators  reported  that  their  students  quite  literally  are  "empowered"  through 
practices  such  as  those  promoted  by  AfL  and  they  are  eager  to  disseminate  them  to  other 
teachers  and  schools.  Yet  one  perhaps  unintended  consequence  of  AISI's  success  with 
AfL  is  that  aspects  of  the  PATs  such  as  multiple  choice  items,  time  limits,  lack  of  access 
to  dictionaries  or  "word  walls,"  and  single  chances  to  demonstrate  knowledge  with  pre- 
established  testing  formats  appeared  to  educators  and  students  to  conflict  with  the  new 
culture  of  teaching  and  learning  that  AISI  is  disseminating  in  Alberta  through  approaches 
such  as  AfL.  For  those  educators  who  had  most  internalized  the  research  findings 
associated  with  AfL,  their  work  with  AISI  was  vital  and  urgent  in  advancing  a  broader 
understanding  of  teaching  and  learning  that  they  believed  held  greater  promise  for  those 
children  traditionally  marginalized  by  orthodox  schooling  practices. 


3.  Conclusion 

AISI  is  now  at  the  cutting  edge  of  precisely  these  kinds  of  changes  that  move  beyond 
individual  initiatives  and  opportunities  into  a  deeper  "Fourth  Way"  of  educational  change 
with  a  carefully  calibrated  blend  of  both  innovation  and  sustainability.  Teachers  have 
acquired  new  skills  as  researchers  and  micro-level  policy  makers  who  identify  problems 
in  children's  learning,  collaborate  with  colleagues  to  formulate  potential  solutions,  and 
then  acquire  funding,  skills  and  support  to  put  their  professional  knowledge  to  work. 
Educators  have  side-stepped  the  kinds  of  short-term  strategies  that  lead  to  "gaming  the 
system"  to  get  test  scores  up  and  instead  are  asking  far  more  radical  and  profound 
questions  of  themselves  and  of  their  colleagues.  They  are  challenging  each  other  to  work 
with  students  to  establish  agreed-upon  criteria  for  excellence  in  learning  and  are 
providing  children  with  multiple  venues  for  acquiring  and  demonstrating  excellence. 
They  are  embedding  technology  into  a  repertoire  of  instructional  strategies  that  presumes 
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agency  and  intelligence  on  the  part  of  learners  and  intentionality  and  purpose  on  the  part 
of  educators.  They  have  become  accustomed  to  the  idea  that  their  long-term  vision  of 
educational  change  matters  and  plays  a  role  in  shaping  the  future  policies  of  their 
province. 

The  importance  of  building  ownership  at  having  people  really  part  of  determining  their 
fates  as  a  key  theme  in  AISI  has  given  districts  a  participatory  and  inclusive  dimension 
that  is  a  striking  exception  to  a  more  common  relationship  of  guarded  caution  or  even 
outright  antagonism  that  exists  between  school-level  and  district-level  authorities  in  many 
jurisdictions  (Bryk  &  Schneider,  2002;  Payne,  2008).  In  our  interviews  and  focus  group 
discussions  the  research  team  was  struck  by  the  highly  iterative  and  porous  relationships 
that  characterized  school  and  district-level  staff  engaged  in  AISI  projects.  Teachers  have 
multiple  opportunities  to  advance  project  activities  at  the  district  level  and  central-office 
administrators  are  welcomed  to  participate  in  shaping  the  daily  life  of  schools  in  creative 
and  innovative  ways  that  go  beyond  the  implementation  of  new  mandates  and  programs. 
The  new  emphasis  on  studying  and  then  applying  research  findings  is  creating  a  common 
respect  for  the  intellectual  demands  of  the  teaching  profession  and  the  responsibilities  of 
educators  at  all  levels  to  stay  abreast  of  educational  scholarship. 

The  research  findings  from  these  12  case  studies  indicate  that  AISI  enjoys  enormous 
popularity  among  educators  and  is  credited  with  helping  them  to  advance  their  skills  as 
thinkers,  researchers,  and  practitioners.  At  the  same  time,  some  lacunae  need  attention. 
First,  to  continue  to  advance,  AISI  will  need  to  find  new  ways  to  engage  parents  and 
community  members.  In  most  instances,  educators  readily  acknowledged  that  they  had 
trouble  engaging  parents  of  students  and  especially  those  students  who  were  often  absent 
from  school.  They  welcomed  the  emphasis  on  parent  and  community  engagement  in 
AISI  Cycle  4  and  were  looking  for  new  ideas  to  improve  in  this  area. 

Second,  AISI  may  need  to  find  ways  not  only  to  add  new  opportunities  for  educators  to 
enrich  learning,  but  also  to  advise  educators  to  abandon  projects  that  have  outlived  their 
usefulness  and  are  not  appropriate  for  the  challenges  that  lie  ahead.  One  principal 
expressed  enthusiasm  for  AISI  in  general  but  also  complained  that  "the  common 
comment  from  most  teachers  is  that  we've  brought  too  many  projects.  'Projectitis'  is 
going  on."  While  this  was  not  a  commonly  heard  sentiment  from  educators,  in  this 
particular  district  it  appears  that  district  leaders  needed  technical  assistance  in  developing 
as  well  as  casting  off  projects.  In  such  districts,  the  merits  of  abandonment  may  need  to 
be  considered  by  AISI  leaders  in  the  years  ahead. 

Third,  AISFs  great  strength — educators'  enthusiasm  for  work  that  they  themselves  can 
initiate  and  drive  in  the  way  that  they  believe  best  meets  the  needs  of  their  schools  and 
communities — may  need  to  be  bolstered  with  a  more  explicit  theory  and  practice  of 
leadership  development.  In  many  ways,  a  great  deal  of  ferment  in  relationship  to 
leadership  is  already  occurring  in  AISI  sites.  One  AISI  coordinator  discussed  how 
themes  such  as  "teacher  leadership  research,  the  change  process,  and  looking  at  resistance 
to  change"  were  prominent  themes  in  AISI  team  meetings  in  her  jurisdiction.  "What 
we're  doing  is  trying  to  build  up  a  leadership  capacity  piece.  The  administrative  PLCs 
that  were  established  in  some  of  our  districts  addressed  similar  themes  related  to 
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leadership,  indicating  that  within  AISI,  leadership  is  emerging  as  an  important  topic  for 
future  inquiry  and  development." 

Interviews  with  teachers  revealed  a  remarkable  lack  of  rivalry  about  the  evolution  of  new 
leadership  roles  that  is  sponsored  by  AISI  projects.  There  seemed  to  be  a  tacit 
understanding  that  taking  on  leadership  responsibilities  or  becoming  involved  with 
innovation  was  admirable  and  that  as  professionals,  teachers  are  obliged  to  exercise 
leadership  in  multiple  domains  simultaneously.  Because  teachers'  "privatism"  has  eroded 
with  support  from  AISI,  teachers  are  able  to  support  one  another  in  their  growth  as 
leaders  across  traditional  disciplinary  and  sector  boundaries. 

If  AISI  wishes  to  generate  new  models  of  teaching  and  learning  in  the  future,  it  is  likely 
to  need  a  more  explicit  theory  of  leadership  development  and  capacity  enhancement. 
Some  innovations,  for  example,  may  require  even  more  networking  opportunities  among 
and  across  district  boundaries,  thereby  strengthening  what  complexity  theorists  term  the 
"robustness"  of  decentralized  networks.  Such  innovations  can  challenge  the  norms  of 
groupthink  that  settle  too  quickly  into  established  norms  of  reflection  and  action.  An 
innovative  spirit  expressed  through  new  forms  of  teaching  and  learning  can  ensure  that 
the  school  and  the  district  remain  learning  organizations  that  are  willing  to  tolerate  a 
certain  amount  of  stress  and  pressure  as  part  of  the  necessary  cost  of  growth  and  change. 

The  cumulative  evidence  from  this  qualitative  case  study  is  that  AISI  is  providing  a  vital 
catalyst  to  Albertan  education  that  far  exceeds  its  actual  funding.  The  universality  of 
educators'  appreciation  for  AISI  as  a  change  network  is  impressive  and  cuts  across  roles 
and  relationships.  The  topics  of  parent  and  community  engagement,  abandonment  of 
themes  and  strategies  that  are  no  longer  priorities,  and  an  explicit  theory  and  practice  of 
leadership  are  three  areas  that  AISI  may  wish  to  address  as  it  now  prepares  for  the  fourth 
cycle.  These  topics  should  be  seen  in  light  of  AISFs  impressive  achievements  that  have 
promoted  educational  professionalism,  capacity  enhancement,  the  study  and  application 
of  research,  and  the  dissemination  and  generation  of  new  knowledge  throughout  Albertan 
education. 
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Chapter  5:  The  Four  Ways  of  AISI 


By  Andy  Hargreaves 

1.  Introduction 

In  today's  rapidly  changing  and  volatile  world,  it  is  no  longer  enough  to  achieve  and 
exemplify  world-class  standards  and  even  to  be  an  international  leader  on  measures  of 
educational  attainment  -important  and  admirable  as  these  things  are.  Learning  more  and 
learning  better  are  no  longer  enough  -  even  and  especially  in  one  of  the  highest 
performing  systems  in  the  world.  This  is  a  time  when  we  must  also  start  to  learn 
differently.  As  my  colleagues  and  I  have  seen  in  our  assessment  for  OECD  of  Finland's 
much-lauded  educational  system,  what  makes  a  system  successful  by  current  criteria  does 
not  guarantee  sustainability  of  that  success  when  the  criteria  start  to  change  (Hargreaves, 
Halasz,  &  Pont,  2008). 

A  successful  system  is  a  learning  system.  It  anticipates  its  way  forward  and  responds  to 
the  challenges  of  the  future.  Sometimes  we  have  to  fix  things  before  they  are  broken.  We 
have  to  start  running  the  race  differently  even  when  we  feel  we  are  way  out  ahead.  This  is 
particularly  true  when  the  terrain  starts  to  change.  In  the  21''^  Century,  the  terrain  is 
changing  in  seismic  proportions  in  almost  every  aspect  of  our  lives.  The  result  is  four 
major  change  imperatives: 

•  the  aftermath  of  global  economic  collapse  that  has  created  an  economic 
imperative  of  developing  2 1  st  Century  skills  for  an  innovative  and  creative 
economy  and  of  doing  so  with  fiscal  prudence  (El  Elrian,  2008); 

•  the  spread  of  excessive  affluence  that  has  reduced  the  quality  of  most  people's 
lives,  and  given  rise  to  the  social  imperative  of  developing  better  lives  and 
wellbeing  for  all,  especially  at  a  time  of  growing  cultural  and  linguistic  diversity 
(Wilkinson  &  Pickett,  2009;  UNICEF,  2007); 

•  the  impact  of  climate  change  and  energy  crises  that  threaten  the  survival  of  our 
species  and  that  raise  the  ecological  imperative  of  producing  both  innovative 
technological  solutions  as  well  as  changes  in  lifestyles  towards  education  for 
more  sustainable  living  (Capra,  2005;  Giddens,  2009). 

•  the  generational  renewal  of  the  workforce  with  the  Boomer  generation  being 
replaced  by  Generations  X  and  Y  whose  approaches  to  life  and  leadership  are 
more  swift,  assertive,  direct,  team-based,  task-centered  and  technologically  savvy 
raising  the  generational  imperative  of  capitalizing  on  these  assets  to  develop  and 
distribute  more  skilled  and  responsible  leadership  in  our  schools  (Howe  & 
Strauss,  2000). 

All  system-wide  educational  change  efforts  must  address  these  imperatives.  Yet  there  is 
no  single  approach  to  educational  change  and  reform.  Approaches  vary  across  the 
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country,  internationally  and  over  time.  They  can  and  do  address  the  four  imperatives  in 
different  ways  and  with  different  degrees  of  adequacy. 

There  have  been  four  general  approaches  to  or  ways  of  educational  and  social  change  in 
many  of  the  developed  countries  in  the  past  50  years  (Hargreaves  &  Shirley,  2009). 
Though  the  identification  of  these  Ways  is  solidly  grounded  in  social  theory  (Giddens, 
1999),  and  also  in  the  findings  of  my  own  research  on  educational  change  over  time  in 
Canadian  and  US  high  schools  (Hargreaves  &  Goodson,  2006),  for  pedagogical  purposes, 
these  Ways  can  be  likened  to  the  mythical  properties  commonly  attributed  to  the  four 
planets  of  the  inner  solar  system:  Venus,  Mars,  Mercury  and  Earth. 

This  chapter  reviews  the  different  reform  directions  represented  in  these  four  Ways  of 
Change,  and  assesses  their  strengths  and  limitations  in  relation  to  the  four  imperatives  of 
the  early  2V^  century.  It  then  examines  in  what  ways  AISI  resembles  aspects  of  these 
Ways  in  its  past  or  present  practices  and  in  its  future  potential.  By  undertaking  this 
exercise,  we  will  then  be  able  to  determine  more  carefully  what  potential  AISI  can  draw 
on  and  what  fears  and  doubts  it  can  reignite  from  past  periods  or  Ways  of  reform  with 
which  it  appears  to  bear  resemblances.  We  can  see  where  it  fits  with  and  where  it  departs 
from  other  reform  patterns  -  what  strengths  it  can  build  upon,  what  flaws  it  might  be  in 
danger  of  repeating,  and  in  what  ways  it  can  and  should  take  the  lead. 


2.  The  Four  Ways  of  Change 

2. 1  The  First  Way  of  Venus 

The  First  Way  of  Venus  stretched  from  the  end  of  the  Second  World  War  and  especially 
from  the  1960s  to  the  mid  1970s.  Economist  John  Maynard  Keynes  and  his  followers 
presented  investment  in  state  services  and  welfare  safety  nets  not  just  as  a  social  good  but 
also  as  a  benefit  for  the  economy  as  it  developed  the  pools  of  talent  that  would  fuel  future 
prosperity.  There  was  immense  confidence  in  the  state's  ability  to  solve  social  problems, 
fueled  by  a  booming  economy  and  spurred  by  the  rising  Baby  Boomer  population. 

In  the  latter  years  of  this  age,  a  rebellious  and  creative  spirit  entered  public  schools  in  the 
form  of  experimentation,  innovation,  and  child-centered  or  progressive  teaching. 
Teachers  and  other  state  professionals  had  great  autonomy  in  the  First  Way.  They 
enjoyed  high  levels  of  passive  trust  from  an  increasingly  prosperous  public  and  were 
largely  left  alone  to  get  on  with  the  job. 

Teachers  today  are  sometimes  nostalgic  for  the  freedom  to  develop  curricula  to  meet  the 
varying  needs  of  their  students  as  part  of  a  mission  to  change  the  world  but  some  of  them 
bemoan  the  loss  of  their  professional  autonomy  more  because  they  could  teach  their 
subjects  just  as  they  chose  (Goodson,  Moore  &  Hargreaves,  2006).  The  First  Way  of 
Venus  therefore  suffered  from  huge  variations  in  focus  and  quality  of  provision  and  also 
standards.  Teaching  was  improved  largely  intuitively  and  individually,  through 
improvisation,  on  the  job. 
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Teachers  from  this  period  remember  their  principals  as  larger  than  life  characters  who  left 
their  stamp  on  the  school,  but  not  always  in  a  good  way.  The  kind  of  school  in  which 
teachers  taught  therefore  depended  on  this  lottery  of  leadership  within  an  unregulated 
profession  of  little  and  only  local  accountability.  The  First  Way  brought  innovation  but 
also  inconsistency.  There  was  no  leadership  development  or  professional  development  to 
create  widespread  consistency  of  impact  or  effort.  Love  was  not  enough. 

The  First  Way  left  a  legacy  of  the  importance  of  innovation  but  within  and  between 
schools,  innovation  occurred  only  in  scattered  islands  -  not  in  continents  or  even 
archipelagoes.  Leadership  made  the  biggest  difference  to  the  success  of  innovation,  but 
good  leadership  was  not  a  focus  of  investment.  It  was  a  matter  of  luck  or  chance.  Trust  in 
educators  as  professionals  was  real;  but  sometimes  this  trust  was  blind  or  misplaced. 


The  Alberta  Initiative  for  School  Improvement  is  about  innovation  as  well  as 
improvement.  It  supports  local  changes  and  initiatives,  compared  to  centralized  or 
standardized  mandates.  It  invests  high  trust  in  the  professional  judgments  of  teachers  and 
principals.  There  are  many  resonances  of  the  First  Way  of  Venus  in  the  orientations  of 
AISI.  More  mature  educators  still  in  the  system  who  are  nostalgic  for  the  professional 
freedoms  of  the  '60s  and  '70s  can  value  AISI's  resurrection  of  a  professional  culture  of 
self-directed  innovation  and  even  improvisation,  and  its  reigniting  of  their  passion  for 
teaching.  They  might  regard  it  as  a  recovery  of  respect  for  professionalism  that  had  been 
lost. 

At  the  same  time,  morose  and  gloomy  educators  (of  whom  the  research  team  sometimes 
heard  but  never  directly  met)  may  recall  the  incoherence  and  inconsistencies  of  the  1960s 
and  1970s  and  have  a  saturnine  skepticism  towards  locally-driven  initiatives  like  AISI 
instead.  An  overly  enthusiastic  celebration  of  self-determined  innovation  and  of 
qualitative  rather  than  quantitative  approaches  to  accountability  also  always  runs  the  risk 


First  Way 

(Venus) 


Trust 


Figure  5. 1  The  First  Way  of  Venus 
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of  triggering  anti-nostalgic  sentiments  towards  unfocused  1960s  and  '70s  idealism  and 
the  discredited  philosophy  of  letting  a  thousand  flowers  bloom. 

Cycle  1  of  AISI  embraced  much  of  the  individual  school  innovation  that  characterized 
the  First  Way  of  change,  but  also,  as  in  the  First  Way,  it  seemed  to  achieve  insufficient 
coordination  among  these  many  bottom-up  initiatives.  The  challenge  for  AISI,  always,  is 
how  to  recapture  the  innovative  spirit  that  characterized  some  of  the  best  practices  of  the 
First  Way,  without  repeating  or  retreating  to  the  excessive  amounts  of  inconsistency,  the 
lack  of  demonstrable  standards  or  clear  accountability,  and  the  absence  of  consistently 
high  leadership  capacity  that  were  also  widespread  throughout  that  period.  AISFs  ever- 
present  challenge,  in  this  respect,  is  how  to  balance  and  reconcile  professional  creativity 
and  organizational  complexity  with  system  coherence  and  accountability. 

2.2  The  Second  Way  of  Mars 

Because  of  the  onset  of  economic  stringency,  and  of  tougher  questions  therefore  being 
asked  about  value  for  money  in  the  use  of  tax  dollars,  a  Second  Way  followed  of  market- 
competition  and  standardization  as  developed  nations  headed  into  the  1980s  and  '90s. 
With  long  gas  lines  and  economic  recession,  coupled  with  a  maturing  labor  force  in 
teaching  that  was  becoming  more  expensive,  people  began  to  question  whether  the  state 
was  still  the  answer  to  everything.  Many  Anglo-American  nations  placed  schools  in  more 
competitive  systems  of  market  choice  for  students  and  their  parents.  The  currency  of  this 
market  was  more-and-more  detailed  standards,  linked  to  high  stakes  tests  that  were 
widely  publicized  in  league  tables  of  performance  and  often  combined  with  weakened 
levels  of  resourcing,  and  accelerated  timelines  for  implementation.  Planning  and 
implementation  processes  became  more  systematic  and  detailed,  and  driven  from  the  top. 
Charter  schools  and  their  international  equivalents  also  began  to  emerge,  though  less 
extensively  in  Alberta  than  south  of  the  border. 

Some  benefits  of  common  standards,  increased  accountability  and  inter-school 
competition  were  evident  in  the  emergence  of  clearer  focus,  greater  consistency,  and 
attention  to  all  students  with  a  stronger  sense  of  urgency.  Outside  Alberta,  drawbacks  of 
the  Second  Way  of  Mars  also  began  to  emerge  quite  quickly  (Hargreaves,  2003).  While 
achievement  gains  often  occurred  for  a  year  or  two,  they  soon  reached  a  plateau.  Parents 
had  more  choice,  but  it  was  the  affluent  ones  who  knew  how  to  work  the  system  to 
advance  their  interests  and  protect  their  privileges.  The  passive  trust  of  the  First  Way  was 
replaced  by  active  mistrust  between  teachers  and  the  public  in  the  Second.  Standards 
raised  the  bar  but  shortfalls  of  professional  support,  due  to  diminished  educational 
investment,  did  not  help  children  reach  it.  The  costs  to  the  quality,  depth  and  breadth  of 
children's  learning,  to  increased  dropout  and  reduced  innovation,  as  well  as  to  the  caliber 
of  teachers  and  leaders  the  profession  could  recruit  and  retain  were  considerable  (Nichols 
Si  Berliner,  2007;  Oakes  &  Lipton,  2002;  New  Commission  on  the  Skills  of  the  American 
Workforce,  2007). 

In  the  Second  Way,  leadership  was  seen  as  overloaded,  unattractive  and  excessively 
exposed  in  the  context  of  punitive  accountability.  Leadership  had  turned  into  line 
management.  Teachers  saw  their  leaders  as  managers  who  had  forgotten  how  to  lead. 
Their  principals  rotated  in  and  out  of  schools  with  increasing  frequency  and  seemed  to 
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have  more  attachment  to  implementing  government  priorities  or  advancing  their  own 
careers  than  serving  their  own  schools  locally  (Hargreaves,  2003;  Fink  &  Brayman, 
2006). 

Alberta  was  arguably  the  first  province  to  enter  the  Second  Way  in  Canada.  Its  Student 
Evaluation  Branch  (now  called  Learner  Assessment)  developed  extensive  and 
sophisticated  instruments  of  testing  and  accountability.  At  the  same  time,  the  teaching 
profession  has  been  more  extensively  and  inclusively  involved  in  designing  and 
implementing  the  provincial  achievement  tests  (PATs)  than  in  any  other  jurisdiction  in 
North  America. 

There  are  undoubtedly  tensions  between  the  First  Way  feel  of  AISI  in  its  promotion  of 
locally  developed  innovation  and  improvement  efforts  in  relation  to  a  wide  range  of 
outcomes,  and  the  Second  Way-like  emphasis  of  the  provincial  accountability  system  and 
its  support  for  PATs  connected  to  a  more  closely  defined  set  of  outcomes.  In  this  review, 
we  found  that  the  closer  to  the  classroom  that  educators  are  in  their  roles  and 
responsibilities,  the  less  supportive  they  are  likely  to  be  of  the  PATs.  The  views  of  senior 
district  staff  are  the  most  favorable,  the  reviews  of  principals  are  more  mixed,  and  the 
responses  of  classroom  teachers  are  consistently  critical.  PATs  can  give  administrators 
leverage  with  teachers  and  change  and  they  can  also  enable  them  to  respond  to  a  range  of 
stakeholders  whom  they  must  routinely  meet.  Classroom  teachers,  by  contrast,  sometimes 
acknowledge  the  need  for  PATs  or  something  like  them,  but  regard  their  presence  within 
AISI  as  being  one  of  distortion  of  and  distraction  from  the  different  learning  and 
improvement  goals  that  they  feel  AISI  projects  are  trying  to  achieve. 

In  general,  the  quantitative  component  of  this  multiple  perspectives  review  finds  no 
strong  association  between  AISI  project  initiatives  and  PATs.  Reassuringly  perhaps,  at  a 
time  of  significant  population  changes  in  many  of  Alberta's  schools,  there  is  no  evidence 
of  AISI  being  associated  with  any  deterioration  in  PAT  scores,  and  indeed,  there  are 
indications  that  AISI  may  be  related  to  small  PAT  gains  in  subjects  like  mathematics  at 
times  when  the  provincial  PAT  average  has  been  falling. 

Overall,  the  rigorously  conducted  quantitative  analysis  concludes  that  it  is  extremely 
difficult  to  demonstrate  associations  of  any  kind  between  AISI  projects  or  AISI  as  a 
whole  on  the  one  hand  and  PATs  on  the  other.  This  is  because: 

•  those  associations  that  have  been  uncovered  often  appear  to  be  statistical  artifacts 
such  as  regression  to  the  mean; 

•  there  is  wide  variability  in  AISI  projects  or  treatments  which  makes  it  hard  to 
group  them  together; 

•  AISI  is  now  so  deeply  embedded  in  and  hard  to  disentangle  from  the  overall 
policy  and  practice  of  the  Alberta  education  system  that  it  is  hard  to  isolate  AISI 
as  an  independent  effect; 

•  many  AISI  projects  goals  and  emphases  are  not  directed  at  increases  in  PAT 
scores,  or  at  improvements  in  subjects  or  areas  of  learning  that  the  PATs  measure; 


66 


The  Learning  Mosaic 


•    PAT  data  are  collected  in  other  ways  for  other  purposes  than  those  associated 
with  AISI  projects,  making  it  difficult  to  track  AISFs  impact  on  particular  cohorts 
of  students. 

In  conjunction  with  the  four  imperatives  that  opened  this  chapter,  the  results  of  the 
quantitative  analysis  raise  significant  questions  about  the  prominence  and  pre-eminence 
of  PATs  within  the  provincial  system  of  Alberta  education  as  a  whole,  and  about  the  role 
and  influence  of  PATs  within  AISI  and  AISI  projects  in  particular. 

As  we  come  towards  the  end  of  the  first  decade  of  the       century,  there  is  evidence  of  a 
worldwide  movement  in  the  developed  countries  away  from  current  levels  of  system- 
wide,  standardized  testing.  In  some  places,  such  as  Nova  Scotia,  within  the  context  of  an 
economic  recession,  elimination  of  system-wide  standardized  tests  is  a  simple  result  of 
cost-benefit  analysis:  the  tests  are  seen  as  "not  worth  the  costs"  (Hargreaves  &  Shirley, 
2009).  In  others,  alterations  to  the  system  are  seen  as  essential  in  order  to  develop  the 
greater  creativity  and  innovation  in  teaching  and  learning  that  are  integral  in  developing 
21^^  century  skills  for  an  advanced  knowledge  economy.  This  is  evident  in  US  moves  to 
develop  different  testing  systems  and  testing  items  (New  Commission  on  the  Skills  of  the 
American  Workforce,  2007)  and  also  in  Finland's  commitment  to  testing  and 
accountability  through  the  use  of  confidential  monitoring  by  samples  in  order  to  preserve 
the  flexibility  and  professional  trust  central  to  remaining  the  world's  most  competitive 
economy  (Hargreaves,  Halasz,  &  Pont,  2008;  Sahlberg,  2006).  Finally,  system-wide 
standardized  testing  has  been  abandoned  in  Wales,  and  has  seriously  abated  to  the  point 
of  almost  complete  elimination  in  England  due  to  mounting  parental,  public  and 
professional  criticism  about  the  impact  of  testing  on  enjoyment  of  and  engagement  with 
learning  and  school,  especially  among  younger  children  (BBC,  2009).  The  recent  passing 
of  Motion  503  in  the  Alberta  legislature  shows  movement  in  the  same  direction. 

Within  AISI,  PATs  are  widely  used  as  measures  of  accountability  and  indicators  of 
impact.  This  seems  to  be  more  because  they  can  be  easily  accessed  and  rapidly  employed, 
than  because  they  are  regarded  as  a  useful  valid  measure  of  project  impacts.  The 
extensive  accountability  requirements  of  AISI  are  seen  as  overwhelming  by  many  AISI 
schools.  An  easily  used  measure  can  be  a  seductive  option  for  busy  teachers,  even  when 
it  is  less  appropriate  than  a  more  time-consuming,  self-developed  one.  At  the  same  time, 
the  review  team  did  not  encounter  systematic  efforts  to  train  teachers  and  schools  in 
designing  valid  and  reliable  indicators  of  progress  that  were  more  coherently  related  to 
their  own  project  goals.  One  possible  way  forward  may  therefore  be  to  build  the  capacity 
of  teachers,  schools  and  districts  to  develop  and  deploy  their  own  rigorous  instruments  of 
accountability  as  a  way  of  inquiring  into  project  impact  and  also  meeting  their 
accountability  requirements  externally. 

Last,  as  the  Second  Way  of  Mars  leaves  a  legacy  of  widespread  use  of  standardized 
achievement  testing  as  a  method  of  assessing  progress  and  impact,  it  also  leaves  a  further 
legacy  of  linear  and  time-bound  processes  of  planning,  development  and  review  linked  to 
cycles  of  funding  administered  from  the  top.  In  the  next  chapter,  Pasi  Sahlberg,  in  his 
response  to  the  reviews,  raises  questions  about  a  project-based  approach  to  AISI,  divided 
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into  three-year  cycles  of  funding.  This,  he  says,  can  tend  to  fragment  AISI  projects  into 
relatively  short-term  initiatives  that  do  not  embed  or  integrate  into  the  system  or  build  on 
one  another  (as  AISI  nonetheless  encourages  project  leaders  to  do).  It  may  also  explain 
why  districts  like  Sumara  and  Davis'  "Arrowhead",  that  have  a  more  top-down  and 
linear,  Second  Way  approach  to  planning  and  implementation,  tend  to  choose  a  focus  for 
AISI  related  to  a  provincial  thrust  and  direction,  backed  up  by  trainers  and  training 
packages. 

Second  Way  thinking  about  planning  and  accountability  may  not  be  well  suited  to  the 
further  development  of  AISI  and  its  culture  of  local  improvement  and  innovation  in 
particular,  or  to  the  worldwide  move  to  develop  more  flexible  school  systems  that  can 
cultivate  the  21"^  century  skills  of  creativity,  innovation,  and  flexibility  that  are  essential 
for  advanced  knowledge  economies.  This  does  not  and  should  not  mean  abandoning 
commitments  to  rigorous  processes  of  planning  and  accountability  in  favor  of  a  return  to 
the  local  inconsistencies  of  the  1970s.  But  it  does  mean  rethinking  how  planning  and 
accountability  within  AISI  and  outside  it  should  be  reconstructed  in  the  more  complex, 
fast-moving  and  innovation-oriented  cultures  and  systems  of  the  21^^  century. 

Second  Way 


(Mars) 


Figure  5.2  The  Second  Way  of  Mars 
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2.3  The  Third  Way  of  Mercury 

A  new  Way  is  needed  that  can  keep  an  eye  on  coherence  and  consistency,  and  retain  the 
sense  of  urgency  about  learning  and  achievement  for  all  students,  while  also  maintaining 
AISI's  positive  impacts  of  professional  growth  and  energy,  as  well  as  developing  the 
higher  levels  of  creative  learning  and  skill  development  that  are  essential  for  competitive 
economies  and  cohesive  societies.  In  this  Third  Way,  2V^  Century  skills  and  systems  take 
on  particular  prominence. 

The  2r^^  Century  skills  agenda  has  its  origins  stretching  back  more  than  thirty  years  to  the 
work  of  Daniel  Bell  (1976)  who  first  invented  the  term  knowledge  society  to  describe  a 
post-industrial  world  that  would  require  an  educated  workforce  capable  of  working  in 
services,  ideas  and  communication.  Since  then,  management  gurus  and  futurists  (e.g. 
Toffler,  1990,  Drucker,  1993),  educational  reform  advocates  (e.g.  Schlechty,  1990; 
Hargreaves,  2003;  Wagner,  2008;  Zhao,  2009),  high  ranking  government  leaders  (e.g. 
Reich,  2001),  national  think-tanks  and  partnerships  (New  Commission  on  the  Skills  of 
the  American  Workforce,  2007;  Partnership  for  21''  Century  Skills,  2009)  and 
international  organizations  (e.g.,  OECD  2001,  2008)  have  argued  for  the  development  of 
21^'  century  skills  in  knowledge  society  schools  that  will  promote  innovation,  creativity, 
flexibility,  adaptability,  problem-solving,  critical  thinking,  lifelong  learning,  ingenuity, 
collective  intelligence,  teamwork,  risk-taking  and  continuous  improvement. 

21^^'  Century  skills  are  part  of  a  Third  Way  of  educational  reform  -  neither  child-centred 
and  permissive,  nor  basic  and  standardized.  Instead  they  are  like  the  winged  messenger  of 
Mercury  -  characterized  by  speed  and  communication  that  suits  a  modern  world  of 
information-driven  profit,  trade  and  commerce.  This  promotes  economically  useful  cross- 
curricular  skills  in  learning;  new  patterns  of  professionalism  as  well  as  professional 
interaction  and  networking  among  teachers:  and  more  rapid  and  flexible  ways  of 
managing  change  in  organizations. 

This  Way  of  Mercury  directly  addresses  three  of  the  four  2V^  Century  imperatives 
outlined  earlier.  It  develops  the  skills  and  processes  that  accelerate  the  innovation  and 
knowledge  circulation  that  are  vital  for  regenerating  a  floundering  economy.  This  culture 
of  innovation  and  ingenuity  is  equally  indispensible  in  dealing  with  the  environmental 
challenges  of  climate  change  and  energy  shortages.  Engaging  with  the  digital  and 
attitudinal  realities  of  2  T' Century  learners  also  appeals  generationally  to  students  and 
their  younger  teachers  who  have  been  bom  digital. 

Although  it  is  important  to  be  realistic  and  acknowledge  that  not  all  the  work  skills  of  the 
2r'  Century  will  be  21^^'  Century  skills  and  that  many  of  today's  middle  class  are 
consigned  more  to  the  routine  cubicle  work  that  is  reminiscent  of  TV's  The  Office,  than 
the  high-powered  corporate  judgment  and  problem-solving  of  The  Apprentice  (Crawford, 
2009),  the  Third  Way  of  Mercury  does  set  a  new  agenda  of  student  skills  and  patterns  of 
judgment  and  decision-making  that  are  required  in  complex,  fast-moving,  flexible, 
information-driven  organizations. 

This  Third  Way  of  educational  and  social  reform  sits  between  and  beyond  the  state  and 
the  market  (Blair  &  Schroeder,  1999;  Giddens,  1999).  In  the  Third  Way,  there  is  stronger 
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support  for  state  professionals  at  the  same  time  as  a  demand  for  more  accountabihty  from 
them.  State  requirements  persist,  and  these  are  now  commonly  expressed  as  market-like 
performance  targets.  Services  offer  more  choice  and  flexibility  for  clients  and  consumers. 
The  most  distinctive  feature  of  the  Third  Way,  perhaps,  is  its  infusion  of  greater  lateral, 
professional  energy  into  the  process  of  change  and  improvement  (Hartley,  2007).  More 
resources  and  energy  are  invested  in  professional  development  and  involvement,  in 
constructing  professional  learning  communities  of  collaboration  and  teamwork  often 
around  data-driven  improvement  agendas,  and  in  constructing  networks  of  professional 
learning  and  interaction  between  and  across  schools  as  ways  to  build  motivation  and 
spread  innovation. 

The  Third  Way  of  Mercury  offers  flexible  and  customized  pathways  for  students' 
learning  in  school  and  for  professionals'  engagement  in  improvement  within  their 
organizations.  Unlike  the  Second  Way  of  Mars,  one  size  no  longer  fits  all.  AISI  offers 
just  such  localized  pathways  of  innovation  and  improvement,  not  in  the  completely  open- 
ended  sense  of  the  First  Way,  but  more  typically  now  in  relation  to  a  guided  set  of 
priorities. 

Since  Cycles  2  and  3,  AISI's  approach  has  been  increasingly  collaborative,  particularly 
within  districts,  and  one  of  the  priorities  of  Cycle  4  is  to  encourage  collaboration  across 
districts  too.  Networks  look  different  depending  on  whether  they  work  on  First,  Second 
or  Third  Way  principles.  First  Way  patterns  of  interaction  are  open-ended  and  not 
especially  accountable  for  results.  Second  Way  networks  often  operate  more  like  clusters 
of  schools  drawn  together  on  training  days  or  for  consultation  purposes  to  facilitate 
implementation  of  government  or  district  priorities.  This  is  how  interaction  across 
schools  seems  to  operate  in  the  Arrowhead  district  discussed  by  Sumara  and  Davis  in 
their  chapter,  for  example.  Third  Way  networks  are  much  more  complex.  They  are 
systems  held  together  or  given  connectivity  by  clear  and  common  purposes,  personal 
trust,  strong  relationships,  and  frequent  interaction  as  in  Sumara  and  Davis'  Pathways 
district. 

All  networks  have  an  architecture  but  not  all  network  architectures  are  equally  effective 
(Hadfield,  2009;  Hargreaves  &  Fink,  2006).  Networks  are  based  around  particular  kinds 
of  conversations  and  activities;  they  have  rules  of  inclusion  and  exclusion;  are  defined  by 
specific  or  less  specific  purposes;  they  are  articulated  by  designed  patterns  of  interaction 
through  meetings,  visits,  website  interfaces,  or  conference  gatherings;  and  they  are 
characterized  by  strong  or  weak  forms  of  accountability  of  participation  and  results. 

I  recently  evaluated  one  especially  effective  network  in  terms  of  its  impact  on 
achievement  results  with  Dennis  Shirley  and  some  colleagues  (Hargreaves  &  Shirley, 
2009).  The  network  comprised  over  300  secondary  schools  that  had  experienced  a  dip  in 
student  achievement  scores  over  one  or  two  years.  It  promoted  improvement  by  schools, 
with  schools  and  for  schools  in  peer-driven  networks  of  lateral  pressure  and  support, 
where  participating  schools  were  connected  with  each  other  and  with  self-chosen  mentor 
schools,  and  invited  to  conferences  that  supplied  them  with  inspiration,  technical  support 
in  analyzing  achievement  data,  as  well  as  a  menu  of  short,  medium  and  long  term 
strategies  for  improving  teaching  and  learning  and  also  achievement  results.  The 
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network's  architecture  emphasized  transparency  of  participation  as  well  as  of  results  in 
which  different  levels  of  performance,  participation  and  success  were  visible  to  all. 

In  Cycle  4,  AISI  plans  to  make  significant  strides  in  networking  across  districts  as  a  way 
to  spread  innovation  and  disseminate  improvements  across  district  boundaries.  As  it  does 
so,  it  would  be  advised  to  move  beyond  First  Way  principles  of  permissive  sharing  and 
celebration  of  practice  alone  to  promoting  more  critical  conversations  among  schools 
about  practice,  and  to  finding  ways  to  honestly  acknowledge  and  identify  different  levels 
of  competence  or  stages  of  development  in  the  initiatives  being  tried  or  in  the  capacity  to 
support  them,  so  that  networks  can  operate  as  clear  and  targeted  forms  of  assistance 
where  needed  as  well  as  mechanisms  to  exchange  ideas  and  strategies. 

Elsewhere,  some  contemporary  system-wide  networks  try  to  meet  accountability 
requirements  by  connecting  network  emphases  and  activities  to  direct  improvements  in 
achievement  results.  In  these  cases,  Third  Way  networking  strategies  are  tightly  tied  to 
the  continuation  and  even  intensification  of  Second  Way  emphases  on  short  term 
achievement  results.  This  connection  can  sometimes  lead  to  a  narrowing  of  focus  in 
networks  to  easily  tested  basics,  or  to  short-term  and  relatively  superficial  interventions 
that  secure  rapid  achievement  gains  -  undermining  longer-term  and  more  ambitious 
efforts  to  transform  teaching  and  learning  for  the  2V^  century.  This  attachment  of 
networking  to  short-term  achievement  results  is  often  linked  to  the  availability  of  data 
systems  that  highlight  gaps  and  difficulties  on  a  just-in-time  basis  and  to  the  pressures  to 
demonstrate  system-wide  results  emanating  from  short-term  election  cycles.  In  recent 
years,  these  kinds  of  network  patterns  have  been  especially  evident  in  England  (Barber, 
2007)  and  Ontario  (Fullan,  2005). 

One  of  Alberta's  great  strengths  is  its  high  degree  of  political  stability  that  it  shares  with 
other  international  high  performers  such  as  Finland  and  Singapore.  This  releases  the 
province  and  also  AISI  from  more  intense  political  pressure  to  demonstrate  the 
immediate  impact  of  initiatives  like  AISI  on  PATs.  Instead,  the  province  can  concentrate 
on  the  longer-term  goals  of  transformation  while  developing  clear  accountable  indicators 
of  the  progress  being  made  in  achieving  them.  However,  the  continuing  organization  of 
AISI  into  three-year  project  cycles  poses  a  risk  to  these  longer  term  goals  and  the  more 
challenging  interactions  and  transformations  they  will  require  from  teachers  -  beyond 
implementing  particular  training  packages,  for  example. 

AISI  and  the  Third  Way  of  Mercury  are  and  should  be  innovative  and  flexible,  as  well  as 
quick  and  agile.  They  also  need  to  ensure  that  they  maintain  a  longer  term  and  sustainable 
vision  as  well  as  a  short  or  medium  term  one,  and  that  they  address  the  needs  and  goals  of 
the  social  and  ecological  imperatives  as  well  as  the  predominantly  economic  ones. 

AISI  and  the  Third  Way  of  Mercury  offer  immense  promise  for  innovation,  improvement 
and  professional  growth  compared  to  the  standardization  and  professional  exclusion  that 
often  characterized  the  Second  Way  of  Mars.  And  they  embrace  creativity, 
professionalism  and  local  determination  without  regressing  to  the  incoherence  and 
inconsistency  of  Venus.  AISI  and  the  Third  Way  can  be  pushed  harder  into  longer  term 
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transformation  for  competitive  knowledge  economies;  into  a  wider  social  agenda;  and 
into  forms  of  robust  accountability  that  extend  beyond  the  existing  province-wide 
instruments  of  standardized  testing.  Such  moves  would  push  AISI  further  into  a  Fourth 
Way  of  change  -  the  sustainable  way  of  Earth. 


Third  Way 

(M£PvCUKy) 


Figure  5.3  The  Third  Way  of  Mercury 

2.4  The  Fourth  Way  of  Earth 

The  Fourth  Way  begins  with  an  inspiring  and  inclusive  mission  beyond  higher  standards 
or  test  scores  alone.  Teaching  and  learning  in  the  Fourth  Way  are  engaging  and  mindful, 
and  so  is  the  learning  of  professionals  (MacDonald  &  Shirley,  2009).  This  learning  is 
often  reflective  and  ruminative.  It  is  not  always  agile  and  quick.  Indeed,  says 
psychologist  Guy  Claxton  (1997),  it  is  this  very  kind  of  learning  that  is  essential  for 
developing  creative  thought. 

Fourth  Way  schools  act  urgently  in  the  present  in  order  to  protect  and  sustain  the  future. 
Their  short-term  goals  are  connected  to  long-term  commitments.  And  the  targets  are 
shared  and  owned  by  them,  not  politically  imposed  from  elsewhere.  As  in  Finland, 
collective  responsibility  precedes  external  accountabihty. 
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The  Fourth  Way  of  Earth  goes  beyond  building  pubhc  confidence  in  education  through 
improved  results.  It  builds  community  with  parents  and  others  in  relationships  of  active 
and  engaged  trust  where  they  work  side-by-side  together  (Obama,  2005,  Hargreaves  & 
Shirley,  2009). 

In  the  Fourth  Way  as  in  the  Third,  teachers  and  schools  work  together,  but  teachers  work 
in  thoughtful,  evidence-informed  communities  that  value  both  hard  data  and  soft 
judgment,  applied  to  deep  and  compelling  questions  of  professional  practice  and 
innovation.  They  do  not  just  hurry  through  meetings  to  produce  just-in-time  reactions  to 
achievement  data.  And  schools  do  not  only  network  with  distant  partners  though  that  is 
an  extremely  valuable  direction  in  itself.  They  also  collaborate  with  immediate  neighbors, 
in  pursuit  of  a  higher  common  good  in  a  community  where  the  strong  help  the  weak. 

Leadership  in  the  Fourth  Way  is  not  individual  but  systemic  (Hopkins,  2007).  Effective 
leaders  help  other  schools.  The  system  provides  resources  to  replace  their  time  when  they 
and  their  key  leaders  assist  their  peers  in  this  way.  This  distributes  leadership  around 
them  and  develops  younger  and  hungry  successors  behind  them.  In  the  Fourth  Way, 
leadership  is  sustainable  as  well  as  successful. 

The  Fourth  Way  of  Earth  meets  all  four  of  the  change  imperatives  that  were  outlined 
earlier  -  economic,  social,  ecological  and  generational.  AISI  is  already  showing  evidence 
of  beginning  to  move  into  the  Fourth  Way  as  well  as  the  Third.  Learning  goals  are  broad 
and  diverse  in  nature.  Principals  are  starting  to  work  together  within  their  districts,  across 
schools,  forming  professional  learning  communities  among  themselves  as  well  as  leading 
them  for  their  teachers.  Cycle  4  is  encouraging  interaction  and  networking  across  district 
boundaries.  More  projects  are  being  encouraged  to  address  parent  and  community 
engagement  and  development.  These  are  all  proven  steps  forward  as  AISI  moves  to  an 
even  higher  level  of  sophistication. 

There  are  still  some  strides  that  AISI  can  take  in  holding  its  place  as  a  world-leader  in 
professionally  inclusive  change: 

•  identifying  schools  and  projects  at  different  levels  of  development  or  capacity; 

•  strengthening  systemic  leadership  across  schools  and  even  districts; 

•  devising  objective  and  transparent  ways  to  highlight  the  need  for  assistance  and 
support; 

•  supporting  teachers  and  schools  in  professionally-driven  efforts  to  develop  robust 
indicators,  accountability  measures  and  shared  targets  that  relate  to  and  are 
appropriate  for  their  own  project  goals. 
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Fourth  Way 

(Earth) 


Figure  5.4  The  Fourth  Way  of  Earth 


3.  Conclusion 

From  the  First  Way  of  Venus,  AISI  has  rekindled  innovation  and  professional  respect, 
but  left  behind  the  inconsistency.  From  the  Second  Way  of  Mars,  it  largely  addresses  the 
urgent  and  focused  emphasis  on  learning  and  achievement  for  all  students,  but  it  does  not 
treat  all  kinds  of  learning  outcomes  as  equivalent  to  tested  achievement  and  it  has  re- 
energized professionals  in  ways  that  the  Second  Way  did  not. 

AISI  has  gained  much  from  and  contributes  a  great  deal  to  the  Third  Way  of  Mercury  in 
terms  of  creativity,  complexity,  innovation,  flexibility,  diverse  pathways,  lifelong 
learning,  teamwork,  networked  interaction  and  working  with  diversity.  By  turning  to  the 
Fourth  Way  of  Earth  and  its  concerns  with  inspiration  and  sustainability,  AISI  can  be 
even  more  effective  in  bringing  people  together,  in  communities  where  leaders  work 
together  across  schools  and  where  the  strong  help  the  weak,  to  serve  a  higher  and 
sustainable  purpose  that  is  greater  than  everyone,  and  to  devise  robust,  school-driven 
evaluation  instruments  as  well  as  set  their  own  improvement  and  growth  targets  together 
that  are  direcdy  connected  to  these  bold  purposes. 

2V^  Century  skills  require  21^^  century  schools  and  systems.  Mindful  teaching  and 
learning;  increased  innovation  and  local  flexibility;  flexible  pathways  of  student  learning 
and  professional  engagement;  evidence-informed  improvement  that  values  data  and 
experience  in  fair  measure;  shared  improvement  targets;  prudent  accountability  on 
measures  that  match  knowledge  society  objectives;  energizing  networks  that  connect 
schools  to  each  other;  and  systemic  leadership  where  leaders  assist  weaker  neighbors  in 
the  service  of  a  greater  common  good  -  these  are  just  some  of  the  strategies  that  will  give 
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us  the  best  21"'  Century  schools  that  will  develop  the  most  challenging  set  of  21"'  century 
skills.  AISI  is  already  a  world  leader  in  many  of  these  strategies.  Its  challenge  now  is  to 
become  a  connoisseur  of  all  of  them. 
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Chapter  6:  AISI:  A  Global  Perspective  

By  Pasi  Sahlberg 

1.  Introduction 

Improving  education  systems  and  their  schools  is  a  global  phenomenon.  Even  the  nations 
where  educational  performances  are  celebrated  are  investing  significantly  in  further 
development  of  teachers,  schools  and  districts.  For  instance,  Japan,  Korea  and  Singapore 
are  investing  heavily  in  fixing  the  flaws  that  the  international  comparison  studies  have 
indicated  in  their  education  systems.  In  Europe,  England,  Germany  and  Finland  all  are 
looking  for  ways  to  make  their  schools  more  attractive  to  students  and  responsive  to  the 
emerging  new  expectations  of  their  knowledge  societies.  Despite  these  global  efforts  in 
improving  schools,  there  is  not  one  dominating  design  of  change  but  rather  a  mosaic  of 
approaches. 

In  this  diverse  global  school  improvement  movement,  AISI  is  unique  in  at  least  three 
ways.  First,  it  is  truly  a  system-wide  effort  to  help  most  schools  and  teachers  to  improve. 
It  is  not  limited  to  selected  districts,  schools  or  teachers  like  most  government-run  school 
improvement  campaigns.  Second,  with  its  four  cycles,  AISI  is  an  exceptionally  long-term 
intervention  that  is  atypical  in  educational  development  where  short-term  targets  of 
politics  determine  the  lifespan  of  government  commitment,  especially  funding.  Third, 
AISI  makes  substantial  funding  annually  available  for  innovation  and  improvement  in 
schools.  Funding  ranging  from  $70  to  75  million  CAD  each  year  corresponds  to  about  2 
percent  of  the  operating  education  budget  of  Alberta.  It  is  therefore  a  significant 
investment  in  human  resources  of  Alberta  education  and  unlike  any  known  school 
improvement  initiative  anywhere. 

The  purpose  of  this  chapter  is  to  present  an  international  perspective  to  assist 
understanding  of  the  value  and  effect  of  AISI  and  contextualize  the  analysis  and  findings 
of  the  wider  multiple  perspectives  review.  Through  a  wider  perspective,  we  hope.  Alberta 
Education  will  be  able  to  continue  good  practices  and  new  cultures  built  in  and  by  AISI 
and  weigh  some  alternatives  for  the  future.  The  challenge  is  that  there  are  only  a  very  few 
large-scale  examples  of  school  improvement  that,  when  compared  to  AISI,  would  do 
justice  to  it.  What  follows  therefore  is  rather  a  reflection  on  AISI  using  an  international 
education  policy  perspective.  It  is  based  on  a  literature  review,  three  commissioned 
studies  of  different  aspects  of  AISI,  discussions  in  the  AISI  Colloquium  2008  in 
Edmonton,  and  in  the  AISI  Review  Retreat  in  Boston  in  August  2009. 

2.  The  Modus  Operandi  of  AISI 

The  purpose  of  AISI  has  been  to  improve  student  learning.  This  has  been  achieved  by 
financing  about  1,700  projects  based  on  the  needs  of  districts,  schools  and  the  people  who 
work  there.  This  has  secured  an  approach  that  has  also  taken  into  account  other  local 
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circumstances,  such  as  willingness  to  learn,  capacities  of  schools  and  engagement  in 
other  development  activities.  It  is  therefore  fair  to  say  that  AISI  has  been  designed  and 
implemented  as  a  meta-project  that  constitutes  several  specific  projects. 

A  project-driven  approach  has  its  place  when  the  targeted  problem  is  clearly  defined  and 
is  soluble  with  available  tools  and  resources.  By  definition,  a  project  has  a  beginning  and 
an  end.  Normally,  between  these  two  is  a  predetermined  set  of  actions  that  make  up 
implementation  of  the  project.  Indeed,  project  logic  emphasizes  planning  and  evaluation 
that  normally  require  significant  amounts  of  documentation,  meetings  and  administration. 
Projects  certainly  have  their  place  in  development  in  general  as  well  as  school 
improvement  in  particular.  But  they  also  have  their  limitations. 

School  improvement  at  the  level  of  the  school  and  classroom  only  rarely  has  a  clearly 
definable  beginning  and  almost  never  an  end.  School  improvement  initiatives,  pilots  to 
test  alternative  practical  models  and  research  studies  are  examples  of  'projects'  that  have 
planned  ends  and  beginnings.  AISI  has  been  administered  in  three-year  cycles  within 
which  projects  have  had  to  be  launched  and  closed.  Although  districts  may  now  plan  and 
bid  for  longer  periods,  and  projects  can  build  on  or  extend  previous  ones,  the  three-year 
cycle  of  funding  and  accountability  still  drives  much  of  AISFs  logic  as  well  as 
widespread  perceptions  of  how  to  undertake,  plan  and  complete  change.  This  is  one  of 
the  main  limitations  of  AISI.  It  rarely  happens  that  any  significant  intervention  in  schools 
that  alters  teaching  and  learning  can  have  substantial  and  measurable  impact  on  student 
learning  in  less  than  a  three-year  period.  This  is  a  particular  challenge  for  those 
implementing  such  projects,  i.e.,  schools  and  teachers. 

Another  limitation  of  a  project-driven  approach  is  lack  of  sustainability.  When  a  project 
is  finished,  activities  financed  by  that  project  often  slow  down  or  stop  altogether.  School 
improvement  that  focuses  on  student  learning  requires  sustained  support  for  schools  and 
teachers.  The  risk  related  to  the  project  approach  in  AISI  is  that  some  improvement 
activities  may  be  terminated  by  the  fact  that  the  project  ends  rather  than  because  the  goals 
have  been  accomplished.  There  are  some  aspects  in  school  improvement  that  can  be  fixed 
by  targeted  projects  (e.g.,  ICT  literacy,  developing  new  curricula,  tracking  and  managing 
individual  student  progress,  or  redesigning  science  laboratories)  but  much  essential 
school  improvement  requires  longer-term  engagement.  For  example,  cooperative  learning 
that  has  been  one  of  the  popular  themes  of  AISI  is  such  a  complex  pedagogical  approach 
that  it  requires  more  than  one  project  to  be  properly  integrated  into  teachers'  repertoires 
of  teaching  (Joyce  &  Showers,  2002;  Sharan,  1999). 

Third,  a  project-driven  approach  implicidy  suggests  that  the  nature  of  change  in  schools 
is  linear;  that  intended  outcomes  can  be  achieved  by  selecting  the  right  actions  or  inputs. 
Yet  this  multiple  perspectives  review  demonstrates  that  effective  (and  even  ineffective) 
school  improvement  processes  are  characterized  by  complexity,  not  linearity;  especially 
in  the  creation  of  new  knowledge  and  practices.  For  example,  increased  communication 
and  collaboration  among  teachers,  networking  with  other  schools  and  linking  one's  own 
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'projects'  to  a  global  community  of  schools  may  create  situations  of  new  learning  where 
working  on  the  pre-determined  plans  and  goals  has  less  importance  than  what  the 
emerging  opportunities  offer. 

The  modus  operandi  of  AISI  through  its  targeted  funding  of  enhancing  selected  priority 
themes  makes  it  more  conventional  than  it  might  otherwise  be.  As  an  initiative  this  can 
easily  be  justified.  But  is  a  project-driven  approach  the  most  appropriate  strategic  choice 
for  AISI  in  the  future?  What  implications  does  our  analysis  of  AISI  as  a  complex  system 
have  for  its  design  and  architecture?  Would  an  alternative  strategy  better  guarantee 
sustainability  and  reduce  'waste'  that  administration  now  produces? 

An  international  perspective  does  not  suggest  many  changes  to  what  AISI  is  already 
doing.  But  this  is  not  to  say  that  AISI  should  continue  as  it  stands.  Three  completed 
cycles  of  locally  driven  school  and  system  improvement  have  created  a  foundation  that 
makes  other  approaches  possible.  Teachers  and  schools  in  Alberta  have  clearly  'learned' 
to  deal  and  live  with  projects.  As  in  any  other  project  environment,  some  have  adapted  to 
project  life  so  well  that  reporting  of  accomplishments  can  reflect  more  than  what  was 
actually  achieved.  Some  international  initiatives,  one  of  them  being  the  Aquarium 
Network  in  Finland  in  the  1990s  (Sahlberg,  2008),  that  have  been  publicly  funded,  lead  to 
the  following  ideas  for  the  future  of  AISI. 

•  Gradually  transform  AISI  from  a  project-based  initiative  into  an  open  network. 
This  would  lead  to  a  system  that  would  have  various  clusters  of  school 
improvement  operating  within  one  network  that  is  facilitated,  coordinated  and  led 
by  the  government.  There  can  be  differing  operational  models,  including  projects, 
smaller  initiatives  and  thematic  actions,  within  this  network. 

•  Seek  ways  to  make  both  entry  to  and  exit  from  AISI  more  flexible  for  schools  and 
districts.  Formal  procedures  in  many  international  school  improvement  initiatives 
limit  schools'  access  to  and  exit  from  the  school  improvement  activities.  This  was 
one  of  the  clear  negative  aspects  of  traditional  school  improvement  in  Finland  that 
eventually  led  to  more  open  door  policies  in  educational  development  in  the 
1990s.  AISI  would  benefit  with  its  highly  professional  participants  from  more 
flexible  ways  to  let  teachers,  leaders  and  schools  join  the  network  whenever  they 
feel  there  is  a  need  for  that.  Development  communities  and  networks  can  also 
suffer  from  retaining  members  who  are  passive  or  act  against  ideas  of 
collaborative  development  and  change. 

•  Reward  and  recognize  collaboration  and  networking.  Teachers  too  need 
incentives  in  order  to  do  great  things.  Making  resources  available  specifically  for 
collaboration  and  networking  could  be  one  means  to  promote  cross-district 
collaborative  projects  and  ad  hoc  teacher  networking.  If  collaboration,  sharing 
ideas  and  networking  are  the  key  principles  of  AISI's  modus  operandi,  then  they 
should  also  be  built  into  the  accountability  system  that  would  recognize  good 
performance  on  the  one  hand  and  hold  people  accountable  for  doing  that  on  the 
other. 
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3.  What  is  AISI  Trying  to  Achieve? 

The  AISI  framework  states  that  the  goal  of  AISI  is  to  "improve  student  learning  through 
initiatives  that  enhance  student  engagement  and  performance  and  reflect  the  unique  needs 
and  circumstances  of  each  school  authority"  (AISI  Handbook  for  Cycle  4,  AISI  Education 
Partners,  2008).  AISI  views  learning  broadly  as  being  concerned  not  only  with  knowledge 
but  also  skills,  attitudes  and  behaviours.  If  AISI  is  compared  to  other  government  funded 
development  initiatives,  it  stands  taller  than  many  others  because  of  the  way  it  perceives 
learning  as  a  broader  concept.  In  contexts  where  a  lot  of  measured  data  of  student 
achievement  are  available  (e.g.,  England,  Singapore,  United  States)  learning  is  often 
narrowed  down  to  achievement  on  standardized  tests.  On  the  other  hand,  in  places  where 
such  systematic  data  are  not  easily  available  (e.g.,  Finland,  Sweden,  Estonia),  learning  is 
normally  seen  within  a  more  generic  concept  that  also  refers  to  non-academic 
achievement,  such  as  social  skills,  self-esteem  and  behaviour. 

The  question  of  whether  an  intervention  and  investment  like  AISI  is  making  any 
difference  to  the  quality  of  education  in  Alberta  has  both  political  and  scientific  interest. 
During  the  first  three  cycles  of  AISI,  it  is  understandable  that  this  question  is  partly 
answered  by  looking  at  possible  gains  of  student  achievement  on  PATs.  But  this  may  not 
be  enough  -  or  relevant  -  in  the  future  if  the  initial  question  -  does  AISI  contribute  to 
student  learning  -  is  to  be  comprehensively  answered.  That  is  why  this  report  also 
suggests  widening  the  scope  of  data  collection  and  diversifying  the  research  function 
within  and  of  AISI.  Some  international  efforts  similar  to  AISI  suggest  that  non-academic 
outcomes,  such  as  improved  student  engagement,  teacher  involvement  in  school 
development  or  improved  school  climate,  can  have  an  equally  significant  impact  on  the 
quality  of  work  and  learning  in  school  compared  to  direct  interventions.  The  issue  that  is 
well  known  among  those  in  charge  of  AISI  is  that  some,  if  not  all  of  the  important 
learning  outcomes  are  only  observable  (and  measurable)  after  a  delay  that  bypasses  the 
length  of  AISI  cycles. 

In  Canada,  as  in  all  other  developed  countries,  the  focus  of  education  policies  will 
gradually  shift  to  emphasizing  the  knowledge,  skills  and  competences  that  are  necessary 
to  Uve  a  happy  life  and  succeed  in  the  world  of  work  (OECD,  2008).  A  common  term 
used  in  this  context  is  '21^^  century  skills'.  These  skills,  among  them  problem-solving, 
team  work  and  risk-taking,  are  only  marginally  covered  in  current  curricula  and 
measurements  that  indicate  how  students  succeed  in  school.  In  the  forthcoming  cycles  of 
AISI,  the  role  of  these  21^^  century  skills  for  competitiveness,  sustainability  and  personal 
development  will  likely  be  strengthened.  The  challenge  is  that  teaching  these  skills  is 
much  more  difficult  and  demanding  to  teachers  than  teaching  conventional  academic 
knowledge. 

The  other  dimension  of  this  challenge  is  that  learning  these  new  attitudes  and  skills  is 
dependent  on  the  cultures  of  the  schools,  in  other  words,  how  schools  as  organizations 
operate  based  on  these  attitudes  and  skills.  It  is,  for  example,  difficult  to  imagine  that 
students  would  learn  to  take  smart  risks  in  their  own  learning  if  their  school  reflected  an 
opposite  way  of  operating  for  their  teachers.  In  general,  teachers  will  be  able  to  create 
and  maintain  conditions  for  learning  2V^  century  skills  in  their  classrooms  only  if  they 
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can  practice  and  experience  these  same  skills  in  use  in  their  staffrooms.  This  two-tier 
phenomenon  of  change  in  schools  will  be  a  particular  challenge  for  the  future  of  AISI. 

AISI  has  been  planted  and  then  grown  in  a  particular  educational  and  cultural  soil.  Highly 
professional  teaching  staff,  dedicated  leaders  and  a  prosperous  economic  context  have 
provided  a  close-to-ideal  situation  for  this  particular  kind  of  school  improvement 
initiative  to  become  a  success.  This  would  not  be  possible  without  at  least  some  degree  of 
mutual  trust  within  schools  between  principals  and  teachers,  in  communities  between 
schools  and  parents,  and  in  the  province  between  districts  and  the  provincial  government. 
But  the  political  reahties  have  also  created  an  exceptionally  tight  accountability  system 
that  holds  schools  and  teachers  accountable  for  their  work  through  tests  and  examinations 
mandated  by  the  government.  For  any  external  observer  this  would  invite  a  question: 
How  can  trust-based  school  improvement,  which  respects  local  needs  and  actions  for 
change,  be  put  in  the  same  equation  with  test-based  accountability  that  indicates  an 
undermining  of  teachers'  and  schools'  judgements  of  how  well  their  students  are 
performing?  In  this  sense,  AISI  is  trying  to  build  mutual  trust  in  the  education  system  at 
the  same  time  as  the  system  is  maintaining  distrust  by  suggesting  an  even  tighter  grip  on 
schools  through  accountability. 

Building  trust  in  pubhc  education  is  an  important  purpose  of  AISI.  Looser  control  over 
each  project,  a  softer  approach  to  determining  success  or  failure  of  projects  and  better 
communication  of  what  schools  are  doing  to  their  communities  would  be  possible  means 
to  give  more  space  for  trust  to  flourish.  The  Aquarium  Network  of  Finland  aimed  to  do 
just  that.  Finnish  schools  and  teachers  were  learning  to  be  in  a  new  environment  that 
required  mutual  trust,  openness  and  transparency  to  operate  as  planned.  The  Aquarium 
Network  was  one  of  the  main  avenues  -  together  with  the  school-based  curriculum 
reform  of  1994  -  to  show  how  to  give  schools  their  freedom  as  well  as  responsibility. 
Thus,  AISI  has  a  great  opportunity  to  become  an  even  more  powerful  engine  of  trust- 
building  in  the  Albertan  education  system. 

How  can  AISI  orientate  itself  to  the  question  of  education  for  the  future?  When 
describing  school  improvement  in  Alberta  as  an  investment  McEwen  (2006)  writes  that 

if  school  improvement  is  about  improving  student  learning,  then  we  must  focus 
on  the  future.  What  students  learn  today  must  prepare  them  for  a  future  more  than 
a  decade  hence  when  they  will  take  their  place  as  productive  members  of  society 
both  economically  and  socially,  (p.  15) 

In  other  words,  AISI  has  by  definition  a  futuristic  orientation.  It  is  strong  in  trying  to  help 
teachers  to  teach  their  students  in  ways  that  help  them  in  turn  to  cope  with  an 
unpredictable  world  ahead.  But  AISI  is  focusing  much  less  on  'learning  from  a  future' 
that  would  help  students  to  comprehend  the  complexity  of  the  unknown.  Future-making 
aspects  of  the  AISI  Cycle  4  are  "A  new  focus  on  the  complexity  of  innovation  and 
change  processes"  and  "increased  emphasis  on  the  use  of  digital  technologies  and 
innovative  approaches  for  communicating,  sharing,  networking,  and  disseminating 
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knowledge".  An  opportunity  that  AISI  now  has  to  take  is  a  good  step  forward  that 
strengthens  the  focus  of  AISI  as  a  facihtator  of  future-making  as  part  of  the  new  vision 
for  Alberta. 


4.  The  Weakest  Link:  Involving  Others 

When  AISI  handbooks  describe  the  AISI  framework  and  its  key  characteristics, 
'partnership'  ranks  very  high  with  being  'student  focused'.  Just  to  illustrate  the  level  of 
detail  in  defining  what  partnership  means,  the  Cycle  4  Handbook  (AISI  Education 
Partners,  2008)  states  that  AISI  is 

a  partnership  among  teachers,  superintendents,  trustees,  business  officials, 
universities,  parents,  and  government.  The  AISI  partnership  is  characterized  by 
trust,  collaboration,  and  teamwork  among  the  education  partners  who  share  a 
commitment  to  improving  education  for  Alberta  students,  who  are  the 
beneficiaries  of  this  strong  and  diverse  partnership.  By  working  together,  the 
partners  continue  to  develop  new  relationships,  strategies,  and  practices  that 
provide  long-term  benefits  to  teaching  and  learning  in  our  province,  (p.  5) 

High  priority  given  to  'partnership'  as  a  principle  of  the  AISI  framework  indicates  that  it 
is  also  an  expected  element  of  accepted  projects.  This  is  a  common  feature  with  many 
other  large-scale  school  improvement  initiatives,  for  example  in  the  Accelerated  Schools 
Project  in  the  United  States  and  Hong  Kong,  The  Innovative  Design  for  Enhancing 
Student  Achievement  in  Schools  (IDEAS)  in  Australia,  and  the  Aquarium  Network  in 
Finland.  In  many  international  large-scale  school  improvement  initiatives,  partnership 
refers  to  parental  involvement,  community  engagement  or  school-university  partnerships. 
This  is  how  it  seems  to  be  in  AISI  as  well. 

Case  studies  included  in  this  volume  indicate  that  there  is  room  for  further  collaboration 
even  within  AISI  itself.  First,  the  steering  of  AISI  would  most  likely  benefit  from  larger 
representation  than  it  has  currently.  According  to  the  AISI  Handbooks,  the  AISI 
Education  Partners  Steering  Committee  that  has  guided  the  development  of  different 
cycles,  themes  and  strategies  constitutes  the  following  AISI  Education  Partners:  Alberta 
Education,  Alberta  School  Boards  Association  (ASBA),  Alberta  School  Councils' 
Association  (ASCA),  Alberta  Teachers'  Association  (ATA),  Association  of  School 
Business  Officials  of  Alberta  (ASBOA),  College  of  Alberta  School  Superintendents 
(CASS),  and  University  Faculties  of  Education.  These  are  all  relevant  and  important 
partners.  A  critical  question  would  be:  Where  is  the  voice  of  parents  or  the  third  sector 
(civil  society)  in  AISI  management?  For  example,  the  experience  from  the  Aquarium 
Network  in  Finland  indicates  that  the  role  of  youth  and  sports  organizations  in  steering 
and  implementing  the  network  strategy  were  vital.  It  was  not  only  that  it  was  'politically 
correct'  to  involve  partners  outside  of  the  education  sector  who  work  with  the  same 
young  people  as  schools,  but  these  extemal-to-education  partners  brought  fresh  ideas  and 
new  questions  to  the  work  of  educators. 

The  AISI  partnership  has  resulted  in  the  building  of  trust,  collaboration,  and  teamwork 
among  the  education  partners.  Indeed,  trust  building  is  one  of  the  conditions  for  success 
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in  AISI.  It  is  also  necessary  that  AISI  pays  attention  to  expanding  current  partnerships  at 
the  district  level  by  engaging  more  individuals,  associations  and  business  organizations  in 
AISI  activities.  This  is  important,  as  international  experiences  suggest,  because  the 
drivers  of  renewal  of  education  often  are  outside  of  school.  Changes  in  the  economy  and 
thereby  in  labour  markets,  local  social  issues  and  ongoing  globalization  all  have  a  strong 
influence  in  how  the  role  of  education  shifts  over  time  in  our  societies.  Government  is 
responsible  for  adjusting  education  policies  to  the  needs  of  the  society  but  it  rarely  is  the 
driver  of  educational  change. 

There  is  no  evidence  that  business  leaders  have  played  any  significant  role,  in  general,  at 
the  local  level.  The  perception  that  business  leaders  are  not  systematically  engaged  in  the 
dialogue  of  school  improvement  in  districts  gets  support  from  my  own  experiences  when 
visiting  several  jurisdictions  and  communities  in  Alberta  in  spring  2009.  Meetings  I 
attended  were  not  related  to  AISI  but  more  general  gatherings  for  people  to  talk  about 
their  schools.  With  a  couple  of  exceptions  those  who  represent  local  businesses  or 
employers  were  missing. 

Further  networking  would  probably  be  good  for  AISI  in  the  future.  There  are  two  aspects 
that  could  receive  attention.  First,  the  central  management  of  AISI  could  expand  the 
representation  of  education  partners  by  involving  individuals  from  youth,  sports  or  other 
third  sector  groups  that  have  a  similar  mission  to  AISI  to  help  young  Albertans  to  learn 
better.  This  would  then,  hopefully,  lead  to  wider  stakeholder  engagement  at  the  district 
level.  The  Finnish  experience  in  a  similar  situation  showed  that  it  would  have  been  a 
serious  mistake  to  leave  out  those  associations  and  individuals  who  work  with  the  same 
youth  with  similar  goals.  Second,  there  could  be  more  networking  between  the  districts 
within  AISI.  One  purpose  of  this  lateral  networking  would  be  to  share  experiences  in  how 
to  establish  constructive  dialogue  and  partnership  with  the  communities.  It  would,  as 
international  experience  suggests,  also  by  itself  enhance  professional  development  and 
strengthen  the  professional  learning  communities  in  these  districts  by  collaboration 
among  teachers  and  principals  from  different  districts. 


5.  Energy  Efficiency  and  Sources  of  Renewal 

AISI  is  a  complex  system  as  others  have  described  in  this  volume.  Complex  systems  are 
open  in  the  sense  that  they  "continuously  exchange  matter  and  energy  with  their 
surroundings  (and  so  judgments  about  their  edges  may  require  certain  arbitrary 
impositions  and  necessary  ignorance)"  (Sumara  &  Davis,  in  this  volume).  From  that 
systemic  perspective,  change  in  AISI  can  refer  to  'improvement'  that  operates  as  a 
process  of  renewal.  A  typical  form  of  renewal  in  complex  systems  is  self-organization 
that  spontaneously  arises  as  the  actions  of  autonomous  agents  that  will  be  connected  and 
co-dependent.  In  an  international  perspective  this  is  a  unique  approach  to  analyze  a  large- 
scale  government-funded  school  improvement  initiative. 

A  key  question  for  the  future  of  AISI  is  to  what  extent  it  will  be  seen  as  a  process  of 
renewing  education  in  Alberta,  or  as  retaining  its  focus  on  improving  student  learning  in 
schools.  Renewal  here  refers  to  continuous  systemic  change  of  the  education  system 
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driven  by  the  changing  environment  and  improved  internal  conditions,  including 
resources  (energy)  (Hargreaves  &  Fink,  2006).  Renewal  is  not  possible  based  merely  on 
rational  agendas  -  what  is  required  is  a  provincial  drive  and  a  common,  inspiring  goal. 
Alberta  has  been  successful,  so  far,  in  building  a  prosperous,  well-functioning  society 
with  a  world-class  education  system.  However,  the  key  success  factor  in  the  future  will 
not  be  national  or  provincial  capacity  to  fix  the  system  and  improve  its  parts  but  system- 
wide  renewal  capability.  In  several  analyses  (see  Sahlberg,  in  press)  of  the  progress  of 
Finland  as  a  knowledge  economy,  such  renewal  has  been  identified  as  one  of  the  key 
issues  in  the  national  strategies  of  Finland  during  the  past  few  years. 

AISI  has  great  potential  as  a  system-wide  change  effort  to  act  as  a  source  of  energy  for 
renewal.  If  this  becomes  a  more  focused  intention  of  AISI,  it  is  important  that  it  will 
bring  people  -  teachers,  principals,  students,  parents  and  community  members  -  even 
more  to  the  forefront  of  action.  An  aspect  that  is  important  in  further  'humanizing'  such 
change  initiatives  as  AISI  is  to  minimize  'waste'  that  from  the  point  of  view  of  schools  - 
reducing  unnecessary  administration,  shortening  wait  time  and  optimizing  paper  work.  It 
also  requires  finding  a  good  balance  between  a  rational  education  policy  agenda  and  an 
emotional  as  well  as  symbolic  drive  that  people  find  inspiring  and  stimulating  enough  to 
take  action.  It  is  perhaps  interesting  for  Albertans  to  read  what  Professor  Stable  (2007) 
wrote  about  the  need  of  renewal  for  the  future  of  Finland: 

Renewal  is  always  based  on  people,  their  knowledge,  learning  ability  and 
motivation.  Technology  as  well  as  societal  structures  have  an  integral  role  in  the 
renewal  process  though  they  are  seldom  the  key  drivers.  In  renewal  we  generally 
deal  with  various  dimensions  -  emotional,  cognitive,  organisational,  and  political 
-  but  the  order  of  appearance  and  importance  goes  from  the  emotional  to  the 
cognitive,  (p.  1) 

Emotional  energy  most  often  flourishes  in  communities  where  people  are  together. 
Dialogue  in  communities  can  become  an  act  of  enriching  interaction  where  people  help 
each  other  to  be  their  best  and  rise  further  than  anyone  could  go  alone.  It  is  to  understand 
that  helping  others  to  succeed  is  the  best  strategy  to  be  successful  yourself.  In  order  to 
create  enriching  cultures  in  districts  and  in  their  schools,  there  needs  to  be  trust  and 
mutual  responsibility  in  the  community  of  educators  and  students.  Trust  is  the  foundation 
on  which  a  community  builds  its  being  and  behavior.  When  there  is  no  trust,  people  fall 
through  that  foundation  to  the  stage  of  fear  and  use  their  energy  on  coping  with  insecurity 
and  survival.  An  enriching  community,  therefore,  emerges  in  trust,  security,  and 
togetherness  -  all  core  values  of  education  in  Alberta  and  beyond.  An  enriching 
community  and  interaction  that  drives  it  are  conditions  for  risk  taking,  ingenuity,  and 
creativity.  It  requires  that  each  member  in  the  community  is  valued,  i.e.,  that  each 
experiences  concretely  that  he  or  she  is  important  and  needed  in  the  community  that 
enhances  the  feeling  of  belonging.  A  culture  of  competition  is  often  harmful  in  efforts  to 
build  enriching  community.  Therefore,  school  improvement  should  enable  teachers  and 
students  to  learn  and  create  conditions  for  enriching  interaction.  Figure  6. 1  presents  ways 
to  build  enriching  communities  that  enable  system  renewal. 


84 


The  Learning  Mosaic 


DREAM 


Self  Responsibility 
Figure  6.1  Building  of  Enriching  Communities 

Creativity  in  school  is  only  possible  if  people  -  both  students  and  teachers  -  are 
encouraged  to  take  risks.  This  requires  safe  and  secure  communities  where  mistakes  and 
effort  are  rewarded  as  much  as  achievement.  It  should  be  noted  here  that  Minister  Dave 
Hancock  in  his  speech  to  the  AISI  community  in  early  2009  encouraged  teachers  and 
schools  to  take  risks  and  not  be  afraid  of  failing  or  being  wrong.  This  is  a  very  powerful 
message  for  a  more  creative  and  experimental  AISI.  Trying  new  ideas  in  practice  often 
brings  joy  and  increases  meaningfulness  in  school  when  people  can  fulfill  their  personal 
wishes  and  create  new  ideas  and  experiences  for  themselves  and  others. 

A  creative  culture  of  learning  is  the  ultimate  goal  of  school  improvement.  That  is  more 
than  'improved  student  learning'.  It  is  driven  by  a  personal  goal,  vision,  or  dream.  It  is 
stimulated  by  positive  emotional  energy  that,  in  turn,  is  a  source  of  renewal  and 
sustainable  development.  A  dream  is  a  counterforce  of  fear  (or  nightmare).  That  is  why 
in  school  improvement  we  need  not  only  rational  goals  and  measurable  targets,  but  also  a 
compelling  vision  and  inspiring  dreams. 

I  have  written  elsewhere  that  policies  that  support  school  improvement  should  focus  on 
developing  three  elements:  creative  and  competent  people,  productive  structures  (i.e., 
schools  and  learning  environments),  and  creative  cultures  of  learning  (Sahlberg,  2008). 
AISI  Education  Partners  can  be  confident  that  schools  have  enough  creative  and 
competent  people.  In  other  words,  based  on  this  review  of  AISI,  there  is  reason  to  claim 
that  a  critical  mass  of  'enrichers'  exists  in  school.  Enrichers  are  those  who  know  how  to 
create  enriching  interaction  and  who  can  work  with  leaders  to  build  enriching 
communities  in  districts  and  schools. 

Second,  AISI  already  supports  and  has  created  structures  that  enable  people  to 
communicate,  meet,  and  exchange  ideas  much  more  than  in  any  comparable  large-scale 
school  improvement  initiative.  But  there  is  always  room  for  improvement.  Further 
networking,  cross-district  collaboration  and  involvement  of  the  wider  community  in 
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dialogue  are  good  ways  to  do  that.  Productive  structures  and  Hght-touch  administration 
are  necessary  elements  of  successful  school  improvement.  It  is  important  that  teachers, 
students,  school  leaders  and  others  involved  in  AISI  have  opportunities  for  informal 
interaction  and  communication  outside  of  formal  professional  contexts. 

Third,  as  mentioned  earlier,  AISI  has  promoted  building  trust  in  schools  and  thereby 
reduced  feelings  of  insecurity  and  fear  in  the  school.  Fear,  frustration,  and  hopelessness 
lead  to  development  of  different  coping  strategies  that  suck  energy  from  individuals.  For 
example,  it  is  a  common  phenomenon  in  schools  that  individuals  play  several  coping 
games  in  order  to  survive  in  insecure  situations.  Not  making  a  fool  of  oneself  in  front  of 
others  is  a  typical  coping  strategy.  It  takes  a  lot  of  energy  to  keep  that  strategy  up, 
especially  in  problem-solving  or  creative  situations  that  school  improvement  typically  is 
about. 

Four  common  aspects  that  are  important  for  renewal  and  that  characterize  successful 
implementation  of  AISI  are  the  following: 

1 .  The  school  has  a  vision  or  preferably  a  dream. 

2.  There  is  creativity  in  the  school. 

3.  The  school  is  an  enriching  community. 

4.  The  school  has  a  culture  of  trust. 


AISI  can  lead  the  way  to  build  ideal  cultures  of  learning  in  schools  by  focusing  on 
enhancing  the  use  of  emotional  energy  and  building  enriching  communities  in  schools. 
Figure  6.2  describes  the  tension  between  intentional  states  of  emotional  energy  and 
enriching  community  on  one  side,  and  an  emotional  vacuum  and  impoverishing 
community  on  the  other  side. 
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Figure  6.2  Community-Energy  Tensions  in  School  Improvement 


What  is  the  essence  of  enriching  interaction  in  school  improvement?  The  answer  is  that 
one  can  adopt  the  tone  of  the  other.  As  Himanen  (2007)  has  claimed,  a  test  of  this  is  to 
see  if  you  can  sense  what  the  other  person  has  always  longed  for.  School  improvement 
policies  -  if  the  community-energy  tension  in  Figure  6.2  is  taken  seriously  -  should 
provide  more  room  for  genuine  dialogue  and  collaboration  in  schools  as  expressed  by 
many  experts  (Hargreaves  &  Fink,  2006;  Joyce  &  Showers,  2002).  Inability  to  do  so  may 
have  very  negative  long-term  effects.  Apathy,  low  self-esteem,  and  fear  are  some  of  the 
features  of  a  culture  where  dialogue  and  collaboration  have  vanished  and  are  replaced  by 
monologue  and  isolation.  This  is  not  a  rare  occurrence  in  schools  that  are  driven  by 
internal  competition,  artificial  races  for  excellence  and  a  search  for  higher  standards  that 
may  not  be  what  young  people  need  in  their  lives.  It  is  therefore  important  that  the  future 
of  AISI  gives  proper  focus  to  building  enriching  communities  in  schools  and  driving 
change  through  emotional  energy  released  by  sustainable  leadership. 


6.  Conclusion 

AISI  is  a  shining  star  in  the  sky  of  global  large-scale  school  improvement.  It  clearly  has  a 
lot  to  offer  to  others  who  are  working  on  similar  goals.  It  is  difficult  to  find  anywhere  a 
comparable  change  effort  that  would  be  of  the  scale,  size  and  overall  magnitude  of  AISI. 
International  literature  on  school  improvement  consists  of  many  similar  projects  but  they 
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are  without  exception  focused  research  projects,  restricted  pilot  or  experimental 
initiatives,  or  short-term  government  interventions  (Chi-kin  Lee  &  Williams,  2006; 
Hargreaves  &  Shirley,  2009;  Harris  &  Chrispeels,  2006;  Sahlberg,  2010).  That  is  why  it 
is  difficult  to  compare  AISI  and  its  achievement  to  any  other  government-run  initiative. 

Finland  is  an  interesting  comparison  to  Alberta.  They  have  same  size  of  population,  are 
both  modern  knowledge-based  societies  and  in  many  ways  have  similar  social  values. 
Moreover,  15-year  old  students  in  Alberta  and  Finland  do  equally  well  in  the  OECD 
PISA  surveys.  But  Finland  has  attracted  much  more  international  attention  in  global 
media  as  a  good  example  of  an  education  nation.  Alberta  has  a  lot  to  offer  to  others  as 
well.  Based  on  this  review  it  is  easy  to  suggest  that  Alberta  Education  would  use  AISI 
bravely  as  a  modern,  evidence-based  example  of  a  system- wide  school  improvement 
practice  that  has  also  a  proven  record  of  making  a  difference  in  schools  and  their 
communities.  Finland,  in  turn,  has  only  a  httle  to  tell  others  about  how  school 
improvement  is  arranged  at  the  level  of  the  education  system.  On  the  global  scene,  the 
strength  of  AISI  as  a  change  design  is  that  it  is  system-wide,  builds  on  schools'  own 
initiatives  and  has  an  extraordinary  system  of  collecting  and  using  data  for  monitoring 
and  research. 

Alberta  also  has  a  great  capacity  of  human  and  financial  resources  to  conduct  something 
of  the  magnitude  of  AISI.  Political  stability  and  sustainability,  a  strong  Teachers 
Association  as  an  active  partner  and  well-trained  teachers  provide  a  unique  context  for  a 
longer-term  change  strategy.  What  distinguishes  AISI  from  the  Aquarium  Network 
initiative  in  Finland  in  the  1990s  is  the  strong  commitment  -  both  ideological  and 
financial  -  by  the  Alberta  government  that  Finland  did  not  have.  The  challenge  that 
remains  in  Alberta  is  the  further  development  of  networked  leadership  of  change  that  is 
an  issue  of  school  and  district  leadership  and  leadership  development  more  than  that  of 
central  administration. 
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Chapter  7:  Findings  

By  Andy  Hargreaves,  Robert  Crocker, 
Brent  Davis,  Lori  McEwen,  Pasi  Sahlberg, 
Dennis  Shirley  &  Dennis  Sumara 

1.  Introduction 

This  chapter  brings  together  the  results  of  the  three  evidence-based  components  and  the 
two  historical  and  comparative  reviews  that  comprise  this  multiple  perspectives  review. 
In  the  project  timeframe  of  six  months,  this  review  has  undertaken 

•  a  system- wide  statistical  meta-analysis  of  AISFs  impact  and  of  the  challenges  of 
measuring  AISFs  impact  on  Diploma  Examinations  and  provincial  achievement 
tests  of  student  performance,  and  a  range  of  other  quantitative  measures  of  survey 
results  such  as  satisfaction  levels; 

•  a  study  of  three  contrasting  school  districts  and  their  different  approaches  to 
interpreting  and  implementing  AISI; 

•  a  collection  of  12  condensed  case  studies  of  AISI  projects  across  the  entire 
province; 

•  a  review  of  and  response  to  these  data-driven  analyses  from  the  perspective  of 
international  comparisons  with  similar  initiatives  in  other  high  performing 
countries,  especially  Finland; 

•  a  location  of  AISI,  its  architecture,  impact  and  future  possibilities,  in  relation  to 
four  historically  sequenced  approaches  to  or  "Ways"  of  system-wide  educational 
reform. 


The  results  of  these  components  have  not  just  been  combined  in  an  additive  way,  but  they 
have  been  synthesized  and  integrated  through  cross-team  dialogue  in  telephone 
conference  calls,  email  interaction,  critical  responses  to  all  draft  chapters  by  all  team 
members,  and  a  three-day  project  retreat  to  review  these  drafts  and  responses  and  both 
develop  and  discuss  overall  findings  and  recommendations,  in  August  2009.  Responses 
and  reactions  to  early  drafts  for  AISI  accuracy  were  also  provided  by  those  responsible 
for  AISI  within  the  School  Improvement  Branch  of  Alberta  Education.  In  some  ways,  we 
hope,  the  conduct  of  this  multiple  perspectives  review  has  been  representative  of  the 
rigorously  interactive  and  collectively  committed  processes  and  practices  of  AISI  itself  at 
its  best.  Chapters  of  findings  and  recommendations  are  the  result  of  this  process. 
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2.  Findings 

This  section  summarizes  the  findings  of  the  review.  It  is  organized  by  the  overarching 
questions. 

In  general,  the  research  team  found  that  AISI  constitutes  a  world-class  and  world-leading 
example  of  a  system-wide  educational  strategy.  This  strategy,  designed  by  Alberta 
Education  and  its  partners,  inspires  teachers  and  administrators.  It  enhances  their 
professional  growth  and  enthusiasm.  AISI  seeds  new,  research-informed  practices  within 
local  communities  then  spreads  them  across  districts  and  schools;  and  it  diffuses  existing 
knowledge  as  well  as  creating  new  knowledge. 

AISI  embodies  a  change  process  that  addresses  the  complexity  and  adaptability  necessary 
in  a  fast-moving,  knowledge-driven  world.  It  avoids  the  excesses  of  unregulated  chaos 
and  permissiveness  of  uncoordinated  innovation  on  the  one  hand,  and  of  hierarchical  and 
inflexibly  linear  systems  of  top-down  or  layered  implementation  on  the  other.  It  achieves 
all  this  new  and  ground-breaking  work  with  no  discernible  negative  impact  on  the 
exemplary  record  of  student  performance  as  measured  by  provincial  achievement  tests 
for  which  Alberta  has  become  world-renowned. 

AISI  has  never  been  a  static  initiative.  It  unfolded  in  a  continuous  culture  of  inquiry, 
openness,  reflection  and  adaptation  that  is  rare  among  government-sponsored 
innovations.  The  School  Improvement  Branch  of  Alberta  Education  does  not  merely 
endure  critical  feedback  but  actively  solicits  and  then  rapidly  responds  to  it.  All  projects 
have  onerous  accountability  requirements  and  have  been  subject  to  rigorous  evaluation, 
leading  to  clear  consequences  of  adaptation,  change,  and  shifts  of  focus  or  direction.  This 
review  forms  part  of  the  transparent  learning  culture  that  is  evident  in  many  of  the 
projects  as  well  as  in  AISI's  overall  leadership.  This  continuous  and  deliberate  process  of 
inquiry  and  review  has  led  to  shifts  in  the  culture  and  orientation  of  AISI  over  time.  In  the 
past  decade,  AISI  has  transformed  and  continues  to  transform 

1.  from  a  project-driven  and  initiative-driven  approach  to  a  more  embedded  and 
continuous  change  process  and  strategy; 

2.  from  a  predictable,  time-bound  planning  process  of  uniform  funding  cycles,  to  a 
more  flexible  process  of  planning  and  development; 

3.  from  a  collection  of  disconnected  or  loosely  connected  projects  to  a  province- 
wide  network  of  improvement  and  innovation; 

4.  from  a  change  process  that  has  swung  between  bottom-up  and  top-down 
orientations  in  the  first  two  cycles,  to  a  change  process  that  balances  and 
integrates  these  dynamics  and  also  adds  a  strong,  lateral  peer-driven  change 
dynamic  in  the  third  and  fourth  cycles; 


91 


The  Learning  Mosaic 


5.  from  a  strategy  to  spread  and  embed  existing  knowledge  in  order  to  enhance 
improvement  and  implementation,  to  a  strategy  that  also  creates  new  knowledge 
in  support  of  increased  innovation. 

Our  review  also  identified  some  of  the  limitations  of  AISI  so  far  as  well  as  challenges 
that  it  faces  in  the  future.  For  example,  elementary  schools  have  embraced  AISI  more 
deeply  than  high  schools,  where  teachers'  understandings  of  their  roles  as  experts  in  the 
area  of  academic  content  knowledge  have  made  it  difficult  for  them  to  focus  on  the 
province's  learners  and  their  current  and  future  needs.  AISI  also  needs  to  work  more 
deliberately  on  leadership  development,  and  especially  on  modifying  the  roles  of 
principals  and  other  staff  to  support  the  development  of  teachers.  AISI  projects  can  also 
benefit  from  more  robust  knowledge  dissemination  and  exchange  across  district  lines. 
Finally,  there  is  scope  for  more  explicit  attention  to  the  development  of  stronger 
relationships  with  parents  and  other  community  members.  These  and  other  findings  are 
organized  in  relationship  to  five  overarching  questions,  plus  an  additional  one,  that 
guided  this  study  and  are  elaborated  in  the  recommendations. 

1.  What  is  the  distinctive  theory-in-action  (change  architecture)  of  AISI? 

2.  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI?) 

3.  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

4.  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

5.  Would  the  values  of  AISI  continue  without  funding? 

6.  What  are  the  future  challenges  and  action  points  for  AISI? 

2.1  What  is  the  distinctive  theory-in-action  (change  architecture)  of  AISI? 

AISI  promotes  and  funds  locally-developed,  district-led  innovations  and  improvements. 
It  networks  educators  and  parent  and  community  members  together  through  schools, 
conferences,  and  a  web-based  AISI  Clearinghouse.  To  do  so,  AISI  has  a  four- 
dimensional  architecture: 

1 .  vertical  -  top-down  and  bottom-up; 

2.  lateral  -  project-to-project,  school-to-school; 

3.  radial  -  outside-in  and  inside-out  research  expertise  and  practitioner  inquiry; 

4.  temporal  -  connecting  medium-term  and  longer-term  perspectives. 

AISI  is  a  complex  mixture  of  top-down,  bottom-up  and  laterally-driven  change.  It  is 
guided  by  the  AISI  Education  Partners  Steering  Committee  and  managed  (but  not 
micromanaged)  by  the  School  Improvement  Branch  (SEB)  of  Alberta  Education.  SEB 
works  collaboratively  with  the  AISI  partners  to  set  priorities  and  strategic  directions  for 
each  cycle.  SEB  manages  three-year  project  cycles;  it  further  manages  the  application  and 
approval  process,  coordinates  conferences  and  updates  a  website  Clearinghouse  to  create 
connectivity  across  projects.  SEB  operates  in  a  consistently  transparent,  inclusive  and 
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responsive  way,  with  a  quiet  passion  for  locally-grounded  and  professionally  driven 
change  that  serves  the  public  good  of  all  students.  It  sees  its  role  as  facilitating,  steering 
and  gently  but  firmly  monitoring  and  revising  this  process  over  time. 

From  the  bottom-up,  AISI's  theory-in-action  empowers  educators  to  develop  professional 
and  intellectual  projects  based  on  their  own  locally-created  needs  assessments  and 
subsequent  initiatives  for  self-initiated  change.  Many  of  these  projects  come  from  the 
individual  passions  or  recent  professional  development  experiences  of  teachers  and 
administrators  who  connect  their  initiatives  to  the  priorities  in  the  current  AISI  Cycle. 
Others  -  up  to  40%  per  cycle  -  are  selected  by  districts  in  relation  to  province-wide 
themes  such  as  differentiated  instruction,  professional  learning  communities  (PLCs)  or 
assessment  for  learning  that  are  AISI  priorities  and  also  related  to  a  more  general  policy 
thrust  in  Alberta  Education  as  well  as  to  the  availability  of  external  trainers  who  seem 
able  to  deliver  implementation.  Although  all  projects  feel  local  in  location,  many  are 
nonetheless  provincially  central  in  origin.  Irrespective  of  the  source,  what  matters  in  any 
project  is  the  degree  of  ownership  teachers  and  school  administrators  feel  towards  it. 

A  stance  of  active  trust  (Giddens,  1994,  p.  186)  from  the  School  Improvement  Branch 
responsible  for  AISI  towards  teachers  and  school  leaders  has  produced  hundreds  of 
locally-generated  initiatives  that  have  catalyzed  educators  to  explore  new  routes  to 
teaching  and  learning  that  often  are  precluded  by  more  orthodox  school  reform  strategies. 
AISI  creates  significant  opportunities  for  increased  teacher  leadership  either  for  teachers 
to  be  promoted  to  roles  of  coordinators  in  their  local  district  office,  or  for  them  to  be 
allocated  time  in  the  school  day  to  plan  and  interact  with  colleagues  within  and  across 
local  schools,  without  having  to  abandon  their  passions  and  contributions  as  classroom 
teachers.  AISI  promotes  many  educators'  professionalism  not  by  taking  them  out  of  the 
classroom  but  by  helping  them  to  understand  better  what  is  transpiring  within  the  school. 

The  change  architecture  for  administrators  and  teachers  plays  out  differently  in  AISI 
projects  and  also  varies  over  time.  At  the  beginning  of  Cycle  3,  and  earlier  in  some 
places,  the  emphasis  moved  from  individualistic  and  uncoordinated  projects  at  the 
discretion  of  schools  or  groups  of  teachers  to  an  "umbrella  approach"  in  many  districts. 
As  a  consequence,  the  change  processes  and  projects  became  more  focused  and  also  more 
centrally  driven  and  coordinated.  Sometimes  this  led  to  an  emphasis  on  implementing  a 
particular  strategy  with  the  support  of  trainers  and  of  administration  by  a  district-based 
coordinator.  In  the  worst  case  scenarios,  these  implementation-oriented  strategies  could 
come  across  as  contrived  or  forced  with  infrequent  interactions  other  than  training  events 
leaving  schools  and  projects  with  little  knowledge  of  what  their  peers  were  doing.  On 
other  occasions,  building  consensus  and  utilizing  broad-based  steering  committees  guided 
the  work  and  the  decisions  around  change  projects  in  a  way  that  spread  responsibility  and 
garnered  support  from  participating  educators. 

The  freedom  and  creativity  AISI  extends  to  its  projects  is  accompanied  by  a  strong 
accountability  component  that  requires  all  projects  to  complete  annual  reports  as  well  as 
more  detailed  analyses  at  the  end  of  each  three-year  cycle,  related  to  indicators  and 
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measures  of  impact.  Many  of  these  measures  and  instruments  are  selected  from  those  that 
are  already  easily  available  such  as  provincial  achievement  tests  and  province- wide 
satisfaction  surveys. 

Laterally,  AISI  projects  and  project  schools  are  usually  connected  within  their  district. 
The  degree  of  connection  varies  from  occasional  common  workshops  and  training  days 
with  external  providers,  to  complex  and  continuous  interaction.  Some  districts  hold 
celebrations  at  which  teachers  or  teams  of  teachers  share  poster  presentations  of  their 
work,  inviting  comments  and  critique  from  their  peers.  A  few  districts  have  also  created 
district-wide  professional  learning  communities  for  their  schools.  Annual  provincial  AISI 
conferences  as  well  as  a  number  of  regional  conferences  that  connect  smaller  rural 
districts  create  further  opportunities  for  schools  to  share  and  celebrate  practices  and  learn 
from  each  other's  experiences.  The  AISI  Clearinghouse  provides  transparency  about  the 
nature  of  the  work  that  is  going  on  in  AISI  sites  throughout  Alberta.  Because  educators 
have  easy  access  to  those  sites,  they  can  contact  colleagues  throughout  the  province  who 
are  working  on  similar  issues  and  share  similar  challenges. 

AISI  expresses  two  aspects  of  innovation  -  creative  combination  and  disruption.  Creative 
combination  involves  bringing  together  different  existing  resources  to  solve  new 
problems  -  refining  methods  of  instruction,  combining  existing  approaches  to 
cooperative  learning,  adapting  and  integrating  new  materials  etc.  Abrahamson  (2004) 
argues  that  this  is  a  neglected  but  effective  way  to  undertake  innovation  in  the  corporate 
world.  Second,  innovation  can  be  disruptive  (Christensen,  1999;  Christensen  et  al.,  2008) 
-  a  potential  that  AISI  poses  when  its  collaborative,  networked  processes  of  teacher- 
driven  innovation  pose  problems  and  tensions  for  traditionally  top-down  systems. 

The  creative,  bottom-up  and  lateral  processes  combined  with  central  guidance  and 
support  that  are  embedded  in  AISI's  change  architecture  explain  much  of  the  universal 
enthusiasm  towards  AISI  that  is  expressed  by  project  participants. 

Conceived  three-dimensionally,  AISI  is  not  only  bottom-up,  top-down  and  lateral  in 
nature;  it  also  combines  inside-out  and  outside-in  change  processes  that  penetrate  into  its 
core  and  back  out  again.  Several  districts  had  collaborated  with  university  faculty  at 
various  points  in  their  project  funding  cycles  and  received  assistance  in  designing 
surveys,  studying  student  achievement  data,  and  modifying  assessment  practices.  AISI 
has  made  explicit  the  connection  between  academic  research  and  professional  practice. 
External  stimulation  and  assistance  are  balanced  and  integrated  with  internal  study  and 
reflection. 

Like  all  change  strategies,  AISI  also  has  a  fourth  dimension:  time.  AISI's  three-year 
cycles  establish  longer  timelines  for  change,  action  and  results  than  is  common  in  most 
other  system-wide  reform  efforts.  In  less  stable  political  environments,  these  are  usually 
driven  by  the  demand  for  measurable  short-term  achievement  results.  In  these  other 
cases,  this  culture  of  presentism  (Lortie,  1975)  can  deplete  energy  and  distract  attention 
from  securing  the  longer-term  transformations  in  teaching  and  learning  that  are  more 
appropriate  for  competitive  knowledge  economies.  AISI  largely  avoids  these 
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distractions  through  an  approach  that  is  iterative,  transparent,  and  participatory.  Project 
participants  consistently  praised  AISI  staff  for  their  accessibiUty  and  respect  for  the  on- 
the-ground  reahties  of  teachers  and  school  staff. 

AISI  has  a  unique  architecture  of  educational  change.  In  terms  of  scale,  it  encompasses 
more  than  95%  of  the  province's  schools  and  around  2%  of  the  provincial  education 
operating  budget.  It  has  now  been  in  operation  for  a  decade  and  is  into  its  fourth  cycle. 
The  spread  and  duration  are  greater  than  for  any  other  school-based  or  network-driven 
initiatives  that  are  usually  smaller,  more  local,  temporary  or  self-selected  and  voluntary  in 
nature.  Yet  compared  to  national  or  province-wide  reforms  of  similar  scale,  AISI  is  not 
about  implementing  or  delivering  centrally-determined  mandates,  on  short  timescales,  in 
a  small  number  of  core  and  basic  learning  areas,  that  are  usually  linked  to  tested 
achievement  results.  It  is  complex,  flexible,  professionally  inclusive  and  locally 
adaptable. 

AISI  values  research-based  practices,  but  does  not  exaggerate  their  universal  applicability 
to  all  schools  and  students,  it  does  not  automatically  exalt  them  above  teacher  judgment 
and  knowledge,  and,  in  recognizing  the  value  of  scientifically-based  improvements,  it 
does  not  exclude  the  value  and  necessity  of  new  knowledge  or  innovation  created  then 
diffused  by  teachers  locally  as  well. 

Most  other  systemic  reform  models  are  at  best  two-dimensional.  They  have  top-down 
pressure,  bottom-up  support  and  perhaps  some  lateral  interaction  to  help  implement 
central  mandates.  AISI  is  four-dimensional.  The  bottom-up  and  lateral  improvement 
processes  are  more  vigorous,  but  more  importantly,  there  is  a  more  deliberate  and  robust 
interactive  engagement  between  outsider  and  insider  knowledge,  and  more  openness  to 
mid-term  and  longer-term  improvement  goals  and  processes  than  other  reform  models 
can  normally  acconamodate.  AISI  is  a  national,  international  and  provincial  treasure  of 
improvement  and  innovation.  It  is  worthy  of  preservation  and  recognition.  Its 
architecture  also  needs  further  renewal. 

While  AISI  is  indeed  a  complex  system,  local  districts  vary  in  the  degrees  of  complexity 
they  exhibit  and  have  the  capacity  to  manage.  Central  steering  and  guidance  was  often 
welcome,  but  AISI  funds  were  absorbed  by  some  districts  into  mechanisms  for 
implementing  pre-packaged  priorities  such  as  professional  learning  communities  that 
seemed  to  spread  in  brief  but  intense  outbreaks  across  the  province.  Local  innovation  or 
improvement  could  turn  into  contrived  collegiality  (Hargreaves,  1994).  AISI  funds  were 
sometimes  used  to  replace  existing  core  funding  for  professional  development  instead  of 
enriching  and  extending  professional  development  for  teachers. 

AISI  connectivity  within  districts  is  often  strong,  but  as  yet  it  is  much  weaker  in 
connecting  schools  and  projects  across  districts,  working  with  universities  and  university 
research  communities,  and  connecting  with  partners  other  than  educators.  AISFs 
networks  and  partnerships  are  largely  ones  of  professionals  working  with  professionals, 
not  professionals  also  working  openly  and  enthusiastically  with  other  partners  like 
parents  and  businesses.  The  directions  established  by  Cycle  4  for  increased  networking 
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and  collaborating  with  other  partners  are  in  this  sense  both  timely  and  welcome,  though 
more  could  be  done  by  connecting  with  other  large-scale  networks  internationally  that 
have  records  of  proven  success. 

AISFs  combination  of  outsider  and  insider  knowledge  on  a  province- wide  scale  is 
impressive.  The  gap  between  these  different  kinds  of  knowledge  is,  however,  currently 
too  great.  On  the  one  hand,  external  knowledge  of  "scientific"  methods  of  instruction 
provided  in  packaged  workshops  by  outside  trainers  is  often  persuasive  and  even 
inspirational  (though  not  always  so,  as  when  some  of  the  research  team  heard  that  in 
classrooms,  some  trainers  could  not  persuasively  practice  what  they  preached  in  their 
workshops).  However,  in  this  format,  what  is  usually  presented  is  interpreted  in  the 
context  of  advocacy  of  a  particular  approach  and  market  associated  with  the  trainers.  So 
it  is  vital  to  develop  strong  school  and  district  cultures  that  can  interact  critically  and 
selectively  as  a  community  with  advocacy-based  presentations  of  external  knowledge.  On 
the  other  hand,  educators  in  schools  did  not  seem  to  be  "accountability-literate"  and  were 
often  inclined  to  select  external  quantitative  measures  like  PATs  that  were  not  related  to 
their  project  goals,  and  to  rely  on  descriptive  and  narrative  portrayals  of  a  qualitative 
nature  when  providing  evidence  of  their  own.  There  is  a  great  deal  of  need  and  a  lot  of 
room  for  projects  and  schools  to  be  able  to  develop  their  own  appropriate  evidence  and 
indicators  of  progress  of  a  quantitative  kind  that  are  as  persuasive,  and  indeed  more 
persuasive  and  appropriate  than  externally  available  measures.  Universities  have  been 
able  to  support  the  development  of  that  capacity  for  some  projects,  and  perhaps  the 
current  focus  on  assessment  for  learning  may  increase  accountability  literacy  even 
further,  but  the  room  for  growth  and  support  remains  considerable. 

Last,  on  the  fourth  dimension,  the  timescales  of  change  within  AISI  are  a  refreshing 
change  from  the  short  timescales  of  imposed,  test-based,  system-driven  reforms  in  many 
other  countries  and  provinces.  Imposed,  short-term  targets  in  these  cases  certainly  do 
deflect  attention  from  longer-term  goals  of  a  more  challenging,  transformational  nature. 
But  equally,  mid-term  and  longer-term  goals  in  AISI  projects  are  less  likely  to  be 
achieved  unless  there  are  clear  indicators  of  more  immediate  or  proximal  progress 
towards  those  goals.  This  is  another  case  for  developing  the  capacity  of  schools  and 
districts  to  develop  and  deploy  their  own  indicators  of  shorter-term  progress  towards  their 
ultimate  goals,  and  also  of  the  need  for  persistence  with  projects  and  initiatives  beyond 
funding  cycles  if  there  is  evidence  that  movement  towards  those  goals  is  demonstrable. 

2.2  What  is  the  value  of  AISI?  (What  are  the  values  of  AISI?) 

No  large-scale  educational  project  has  value  unless  it  gets  results.  What  matters  is  what 
kinds  of  results  are  valued  and  what  methods  are  available  or  can  be  devised  to 
demonstrate  them  (or  their  absence).  The  results  and  how  confidently  we  can  make 
statements  about  them  are  the  value  of  AISI,  and  what  things  count  as  results  (and  the 
ethical  means  of  achieving  them)  are  the  values  of  AISI. 

At  its  core,  AISI  seeks  to  "improve  student  learning  and  enhance  student  engagement  and 
performance"  (AISI,  2008).  Its  unique  architecture  is  the  organizational  and  ethical 
methods  it  selects  and  designs  to  try  and  secure  these  results. 
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Positive  changes  over  time  were  found  for  all  measures  in  all  three  AISI  cycles.  AISFs 
impact  on  provincial  achievement  tests  (PATs)  was  small  but  larger  for  local 
achievement  measures  and  survey  measures.  Discernible  effect  sizes  on  PATs  are  rather 
modest,  and  many  seem  attributable  to  being  statistical  artifacts  of,  for  example, 
regression  towards  the  mean  or  outlier  effects. 

AISFs  impact  on  provincial  achievement  tests  (PATs)  is  the  first  and  most  obvious 
measure  that  many  might  choose  to  determine  the  initiative's  effectiveness.  Here,  the 
evidence  is  not  compelling  on  any  dimension  or  in  any  direction.  There  are  few 
consistently  positive  or  negative  correlations  between  AISI  themes  or  treatments  and 
PATs.  Some  upward  trends  could  be  detected  in  mathematics  achievement  at  a  time  when 
provincial  scores  were  falling,  but  even  these  were  not  sustained. 

PAT  results  can  be  interpreted  in  a  number  of  ways.  Perhaps  AISI  has  had  no 
significantly  consistent  effect  anywhere  on  anything  connected  to  student  learning. 
Having  used  several  analytical  methods  to  examine  correlations  with  PAT  scores  of 
project  themes  and  treatments,  across  projects  and  over  time,  the  widespread 
demonstrable  absence  of  anything  in  terms  of  effects,  one  way  or  the  other,  seems  the 
least  likely  explanation. 

A  second  explanation  is  methodological.  PAT  data  are  not  easily  connected  to  traces  of 
particular  students  who  have  experienced  specific  AISI  initiatives.  Apparently  similar 
AISI  projects  on  the  same  themes  might  be  and  clearly  are  sometimes  implemented  quite 
differently  -  so  effects  may  not  be  at  all  consistent.  Projects  vary  over  time  as  they 
mature.  Many  AISI  project  themes  are  also  not  meant  to  have  a  direct  impact  on  PATs. 
They  are  designed  to  change  the  nature  of  learning,  rather  than  raise  scores  in  existing 
kinds  of  learning.  There  is  a  strong  need  to  create  and  identify  instruments  and  indicators 
of  AISI  impact  that  have  a  clear  and  coherent  relationship  to  AISI  project  goals. 

A  third  explanation  is  systemic.  AISI  has  become  increasingly  integrated  into  the 
educational  system  and  improvement  processes  of  the  province  as  a  whole.  It  is  a 
complex  reform,  not  a  simple  treatment  or  intervention,  and  part  of  its  success  is  its 
increasing  influence  on  the  educational  culture  in  general.  Highlighting  its  independent 
impact  is  far  from  easy.  One  promising  step  forward  might  be  to  design  some  AISI 
projects  as  experimentally  controlled  interventions  -  an  idea  supported  by  one  member  of 
the  review  team.  Given  AISFs  culture  of  integrating  and  spreading  ideas  throughout  the 
profession,  on  a  continuous  and  cumulative  basis,  a  second  possibility  is  to  devise  more 
precise  impact  measures  project-by-project,  then  cumulate  these  effects  across  projects. 

AISI  shows  stronger  correlations  and  effect  sizes  with  survey  measures  of  satisfaction. 
Increases  in  student  satisfaction  and  changes  in  attitudes  might  be  seen  as  indicators  of 
improved  levels  of  student  engagement  in  learning  -  something  that  usually  precedes 
improvements  in  achievement.  However,  attitudes  and  satisfaction  are  only  rough 
indicators  of  engagement  and  it  would  be  helpful  for  more  projects  to  use  one  of  the 
many  student  engagement  instruments  that  are  already  available  instead. 
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The  strongest  AISI  impacts  were  on  measures  of  teacher  growth.  It  is  possible  that  these 
represent  a  halo  or  Hawthorne  effect  although  that  in  itself  is  an  indicator  of  teachers' 
appreciation  of  the  trust,  resources  and  recognition  that  are  accorded  to  them  in  the  AISI 
architecture.  Our  review's  qualitative  findings  suggest  something  deeper  is  also  at  work 
in  terms  of  AISI's  impact  on  teachers'  sense  of  professionalism  and  on  the  development 
of  teacher  leadership  opportunities  and  experiences.  These  factors  represent  AISI's 
values  as  much  as  its  actual  value. 

Informants  all  agreed  that  AISI  is  catalyzing  authentic  and  deep  conversations  about 
teaching  and  learning  that  are  contributing  to  a  richer  repertoire  of  instructional  practices 
and  improved  student  learning  in  Alberta.  They  credited  AISI  with  giving  them  new 
ways  to  observe  student  learning,  identify  obstacles  to  achievement,  and  revise 
instruction  so  that  their  students  learn  at  high  levels.  By  exposing  educators  to  alternative 
sets  of  practices,  by  embedding  ongoing  support  into  schools  through  AlSI-funded  lead 
teachers  and  consultants,  by  connecting  teachers  and  projects  to  each  other  in 
relationships  of  mutual  learning  and  support,  AISI  has  helped  to  re-ignite  teachers' 
curiosity  about  new  and  better  ways  of  teaching  their  students. 

Not  all  collaboration,  innovation  and  networking  are  effective  and  one  or  two  examples 
that  came  to  our  attention,  especially  in  some  districts'  interpretations  and 
implementations  of  province-wide  thrusts  and  associated  training  programs  could  be 
superficial,  faddish  or  forced.  There  is  certainly  a  need  to  push  harder,  deeper,  more 
consistently  and  more  urgently  in  ensuring  that  cultures  of  professional  collaboration 
move  beyond  sharing,  celebration,  contrived  implementation  and  quick  exchange  into 
interactions  among  colleagues  that  are  characterized  by  greater  critical  challenge  and 
inquiry.  But  in  general,  research  evidence  points  to  strong  associations  between 
professional  collaboration  and  improved  student  learning  (Rosenholtz,  1989, 
McLaughlin,  &  Talbert,  2001),  and  the  qualitative  testimonies  of  AISI  participants,  as 
well  as  the  quantitative  correlations  with  professional  growth  reinforce  that. 

2.3  Has  AISI  changed  the  culture  of  education  in  Alberta?  If  so,  how  has  it? 

AISI's  change  architecture  has  led  to  clear  shifts  in  the  culture  of  teaching  and 
improvement  in  Alberta.  We  found  many  instances  of  AISI  influencing  school  and 
district  policies  in  ways  that  represented  a  marked  shift  in  understandings  about  teaching 
and  learning  at  the  school  and  classroom  level.  This  was  evidenced  in  the  creation  of 
common  report  cards,  the  alignment  of  curricular  content  with  local  assessments,  and  the 
development  of  principals  as  instructional  leaders  of  learning,  for  example. 

Instead  of  seeing  assessments  as  unwanted  external  impositions  of  provincial 
achievement  tests,  the  emphasis  on  assessment  for  learning  has  helped  teachers  grasp  the 
value  of  diagnostic  and  formative  assessments  that  can  support  their  classroom  practice. 
There  is  room  for  further  growth  in  terms  of  teachers  and  schools  developing  and 
deploying  more  of  their  own  designed  or  chosen  quantitative  instruments  and  indicators 
so  they  can  monitor  impact  of  and  progress  in  their  self-designed  initiatives;  but  the 
emphasis  on  assessment  for  learning  in  AISI's  third  cycle  has  undoubtedly  started  to  lay  a 
foundation  of  a  learning-driven  culture  of  greater  assessment  literacy. 
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One  clear  and  demonstrable  impact  of  AISI  on  the  wider  educational  culture  of  Alberta  is 
in  terms  of  consolidating  and  extending  a  strong  and  enthusiastic  culture  of 
professionalism  and  professional  collaboration  among  schools  and  their  teachers.  Without 
exception,  all  of  the  educators  and  parents  we  interviewed  were  enthusiastic  about  AISI, 
and  the  way  that  it  energized  the  profession.  This  AlSI-driven  culture  has  challenged  the 
three  Ps  that  traditionally  perpetuate  classroom  conservatism  in  the  culture  of  teaching: 

•  presentism  -  concentration  on  short-term  changes  that  make  visible  and 
immediate  differences  with  one's  own  students,  or  that  respond  to  external 
accountability  demands  for  short-term  gains  in  tested  achievement  results  (Lortie, 
1975;  Hargreaves,  &  Shirley,  2009a); 

•  privatism  -  enforced  or  preferred  isolation  from  other  teachers,  and  reliance  on 
individual  judgment  and  improvisation  compared  to  sharing  and  developing 
expertise  with  colleagues  (Little,  1990;  Rosenholtz,  1989) 

•  parochialism  -  believing  that  practical  experience  and  personal  judgment  are 
always  superior  to  external  theory  or  the  evidence  of  research  (Hargreaves,  1984). 

At  the  same  time,  AISI  projects  and  the  culture  deriving  from  them  have  largely  (though 
not  completely)  avoided  the  excesses  of  other  large-scale  reform  efforts  and  their 
attempts  to  counter  the  three  Ps  -  compelling  teachers  to  participate  in  particular  kinds  of 
trainer-driven  and  administratively  imposed  professional  learning  communities 
{contrived  collegiality),  where  teachers  are  required  to  meet  to  look  at,  respond  to  and 
implement  the  results  of  analyzing  externally  produced  data  and  research  {professional 
dependency),  in  relation  to  raising  scores  in  short-term  achievement  results  on  a  narrow 
range  of  tested  subjects  (perpetuating  cultures  of  presentism  and  conservatism). 

With  the  vision  and  freedom  to  become  engaged  with  medium  and  long-term  change, 
classroom  teachers  involved  with  AISI  are  grateful  for  the  opportunity  to  study  research 
and  they  integrate  research  findings  into  their  ongoing  professional  development  and 
classroom  practice.  Schools  have  changed  as  a  result  of  AISI's  work  to  provide  more 
time  and  support  for  professional  development,  and  to  increase  dedication  to 
collaborative  decision-making  involving  a  wider  range  of  participants.  If  there  is  any 
single  area  in  which  AISI  is  most  advancing  policy  changes  at  the  provincial  level  and 
throughout  the  wider  culture  of  education,  it  is  in  this  crucial  domain  of  collective 
learning,  connectivity  among  schools,  and  overall  enhancement  of  capacity. 

Teachers  have  also  acquired  new  skills  as  researchers  and  micro-level  policy  makers  who 
identify  problems  in  children's  learning,  examine  external  research  evidence,  collaborate 
with  colleagues  to  formulate  potential  solutions,  and  then  acquire  funding,  skills  and 
support  to  put  their  professional  knowledge  to  work.  Overall,  more  and  more  teachers 
have  become  accustomed  to  the  idea  that  their  long-term  vision  of  educational  change 
matters  and  plays  a  role  in  shaping  the  future  policies  of  their  province. 
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Along  with  changes  in  teaching  have  come  shifts  in  how  leadership  is  developed  in 
schools.  Leadership  has  come  to  include  and  encourage  greater  teacher  leadership 
(Lieberman  &  Miller,  2004;  Katzenmeyer  &  Moller,  2001)  and  distributed  leadership 
(Harris,  2008;  Spillane,  2006)  -  not  everywhere  and  not  always  -  but  by  taking  on  roles 
as  consultants  and  coordinators  and  by  securing  slots  of  time  in  school  to  coach  and 
mentor  colleagues,  teachers  have  increasingly  spread  their  wings  to  be  leaders  of  other 
teachers.  Leadership  is  no  longer  confined  to  the  principal's  or  superintendent's  office 
but  is  increasingly  being  spread  throughout  the  professional  community,  where  it  retains 
a  close  connection  to  classroom  learning.  This  is  a  significant,  inspiring  and  world- 
leading  aspect  of  the  changing  culture  of  education  in  Alberta,  at  time  when  teacher 
leadership  is  little  more  than  a  cliche  or  a  contrivance  in  many  other  jurisdictions. 

AISI  has  also  helped  combat  conservatism  in  the  culture  of  teaching  and  administration 
by  promoting  a  culture  of  risk-taking.  In  his  remarks  at  the  AISI  Conference  in  February 
2009,  Alberta's  Minister  of  Education,  Dave  Hancock,  communicated  that  mistakes  were 
to  be  expected  and  welcome  along  the  way  to  meaningful  school  change.  Such 
encouragement  was  very  much  appreciated  by  educators  who  were  eager  to  pilot  new 
initiatives  and  to  take  greater  risks  to  reach  disengaged  students. 

Teachers  stated  that  AISI  projects  offered  just  the  right  amount  of  risk  and  reward  for 
those  who  loved  teaching  yet  also  wanted  to  explore  other  dimensions  of  the  educational 
profession.  AISI  enables  teachers  to  develop  new  skills  in  the  areas  of  experiential 
education,  technology  development,  and  local  history  that  may  not  be  directly  linked  to 
gains  on  provincial  achievement  tests  but  nonetheless  have  great  educational  value.  This 
approach  is  integral  to  the  deployment  of  2r^  century  professional  skills  in  a  rapidly 
changing,  culturally  diverse  and  knowledge-driven  society.  It  is  essential  to  a  learning 
mindset. 

There  are  three  clear  ways  in  which  AISI  does  not  yet  seem  to  have  influenced  the  wider 
culture  of  education  and  educational  change  in  Alberta.  The  first  concerns  the  existence 
of  prior  and  parallel  cultures  of  hierarchical  leadership  and  administration  in  a  number  of 
districts.  The  second  is  related  to  the  central  administration  of  Alberta  Education  and  its 
impact  on  school  and  district  cultures.  The  final  limitation  relates  to  the  need  for 
extension  of  networking  activities  across  districts  to  promote  optimal  learning  among 
educators. 

AISI  initiatives  and  the  ways  in  which  they  are  developed  are  often  absorbed  into  the 
existing  cultures  of  administration  within  school  districts,  which  they,  in  turn,  seem  to 
amplify.  Districts  organized  on  hierarchical  lines  with  a  narrative  of  management  tend  to 
decide  on  and  impose  a  focus,  invest  in  external  packages  and  trainers,  use  resources  to 
put  coordinators  into  the  district  office  thereby  swelling  the  ranks  of  administration,  and 
create  little  independent  connectivity  among  schools.  Lines  of  control  are  top-down, 
implementing  administratively  selected  initiatives  and  making  it  difficult  for  schools  to 
learn  from  each  other.  Districts  with  an  ethic  or  narrative  of  service  provide  more  trust, 
invest  more  heavily  in  relationships,  and  secure  commitment  to  common  goals,  but 
patterns  of  implementation  are  still  somewhat  top-down  and  paternalistic,  and  staff 
overload  is  heavy.  This  restricts  the  opportunities  for  organizational  learning. 


100 


The  Learning  Mosaic 


Some  of  the  educators  interviewed  in  the  15  districts  studied  in  the  two  quahtative 
components  of  the  multiple  perspectives  review  communicated  that  Alberta  Education  is 
not  perceived  as  being  part  of  a  wider  learning  community.  It  is  seen  as  a  top-down, 
dissemination-oriented  structure.  So  far,  AISI's  flexible,  adaptable,  participatory  and 
networked  approach  with  its  broad  conception  of  learning  has  interrupted  this  perception, 
but  more  as  a  refreshing  alternative  to  larger  transactional  approaches  that  leave 
educators  wary  of  other  government  initiatives. 

AISI  is,  in  many  respects,  a  change  of  the  Third  and  Fourth  Way  -  flexible,  adaptable, 
participatory,  networked  and  broad  in  its  learning  goals.  But  the  administrative  structure 
of  Alberta  Education  is  still  largely  seen  as  operating  more  on  the  top-down  and  linear 
principles  of  the  Second  Way.  The  dislike  of  provincial  achievement  tests  by  teachers  and 
to  a  lesser  extent  principals  is  undoubtedly  part  of  this,  and  greater  assessment  literacy  in 
the  teaching  profession  as  well  as  the  development  of  a  broader  assessment  portfolio  in 
the  province  may  moderate  these  perceptions  somewhat. 

The  transactional  model  of  Alberta  Education  is  not  unusual  and  seems  to  operate  like 
most  other  education  ministries.  It  is  perceived  by  AISI  participants  as  a  system  of  central 
policy  development  that  is  then  implemented  through  the  hierarchical  authority  of 
individual  superintendents  and  line-managed  by  principals  below  them.  This  system  is 
often  well  organized  to  implement  common  programs  and  strategies.  It  is  less  suited  to 
innovation  and  to  developing  practices  that  require  local  discretion.  On  the  ground,  AISI 
is  in  tension  with  the  existing  policy  culture.  But  as  AISI  progresses  further  and  policy 
goals  also  begin  to  incorporate  more  innovative  elements  suited  to  knowledge  economy 
goals,  this  tension  could  become  a  creative  and  energizing  one  of  productive  disturbance. 

Whereas  AISI  appears  to  be  releasing  the  stranglehold  of  privacy  on  the  culture  of 
teaching  in  Alberta,  there  is  a  persistence  of  privacy  and  isolation  between  districts  that 
inhibits  the  potential  for  learning  and  networking  between  schools  and  projects 
independently  of  detailed  district  control. 

Within  schools,  AISI  appears  to  be  eradicating  the  longstanding  presence  of  privacy  in 
the  culture  of  teaching.  PLCs  among  teachers  and  administrators  have  been  established  to 
study  the  real  and  most  daunting  problems  as  well  as  the  most  inspiring  and  innovative 
challenges  facing  schools  and  then  to  develop  new  strategies  for  responding  to  them. 
These  are  not  just  individual  teacher  opportunities  but  collective  professional 
responsibilities.  This  is  an  enormous  achievement  that  has  eluded  educational  reformers 
in  many  other  jurisdictions  around  the  world. 

The  greater  challenge  of  privacy  and  isolation  that  classroom  teachers  have  experienced 
in  the  past  is  now  a  different  one.  It  is  the  privacy  and  isolation  that  insulates  and 
separates  school  districts.  This  inhibits  the  potential  for  learning  across  schools  and 
projects  independently  of  detailed  district  control.  While  some  districts  have  been  able  to 
surmount  these  problems,  systemic  decisions  now  have  to  be  made  on  behalf  of  teachers 
and  learners  in  the  others: 
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Is  AISI  essentially  going  to  an  outlier  to  or  even  a  safety  valve  for  a  relatively 
traditional  provincial  system  of  education? 

Or,  in  a  context  of  the  province's  reinvention  as  a  competitive  and  innovative 
knowledge  economy  within  an  increasingly  diverse  community,  can  AISI  now  be  the 
catalyst  for  a  more  participatory  and  decentralized  process  of  policy  development? 

In  other  words,  can  AISI  create  a  renewed  and  reinvented  relationship  between  the 
central  ministry  and  its  districts  as  well  as  among  the  districts  themselves? 

None  of  the  people  interviewed  by  the  review  team  were  able  to  point  to  horizontal 
(between  and  among  districts)  or  vertical  (across  levels  of  organization,  from  schools 
through  the  Ministry)  effects.  No  one  was  aware  of  what  was  going  on  in  other  districts 
in  any  great  detail.  What  little  was  known  appeared  to  be  accidental.  Given  the  pockets  of 
remarkable  innovation  in  the  province  and  deep  commonalities  in  interest,  expertise,  and 
activity  across  jurisdictions,  the  time  seems  right  for  inter-district  networking  that  extends 
beyond  the  short-term  encounters  of  annual  conferences  or  the  non-interactive  structures 
of  web-based  archives.  The  more  that  this  networking  develops,  the  more  there  will  be 
creative  and  hopefully  productive  tension  with  the  existing  culture  of  ministerial  and 
district  administration  in  Alberta. 

To  some  extent,  AISI  enables  and  encourages  a  productive  disruption  of  existing  systems 
-  constituting  an  important  form  of  the  disruptive  rather  than  merely  incremental 
innovation  described  by  Clayton  Christensen  (1997;  also  Christensen  et  al.,  2008).  Here, 
disruptive  innovations  like  the  jet  engine  in  relation  to  the  propeller-driven  one,  the  diesel 
shovel  compared  to  the  steam  shovel,  or  laptops  compared  to  desktops,  do  not  introduce 
small  changes  to  existing  systems,  but  bring  about  significant  step  changes  in  the  system 
as  a  whole.  Like  all  complex  systems,  all  the  districts  we  studied  are  engaged  in  ongoing 
adaptation  as  new  educational  and  contextual  challenges  arise,  and  these  adaptive 
activities  certainly  preceded  the  introduction  of  AISI.  However,  by  challenging  districts 
to  innovate,  demanding  accountability,  and  infusing  a  level  of  uncertainty  around  the 
maintenance  of  funding,  AISI  is  providing  a  different-from-usual  source  of 
disequilibrium.  It  is  not  allowing  districts  to  slip  into  a  'comfort  zone'  or  to  do  'business 
as  usual.'  It  is  unlikely  that  this  sort  of  creative  disruption  would  occur  without  AISI. 

2.4  Is  it  possible  for  jurisdictions  to  do  these  projects  and  activities  without  AISI? 

Educators  tended  to  view  AISI  not  so  much  as  the  point  of  departure  for  new  values,  but 
rather  as  an  opportunity  and  funding  source  to  realize  values  they  already  cherished  but 
found  difficult  to  fulfill. 

Districts  needed  funding  to  support  AISI  consultants,  to  provide  teachers  with  release 
time  to  learn  from  their  colleagues,  to  purchase  resources,  and  to  send  teachers  to 
professional  development  activities  such  as  the  annual  conferences  of  the  Alberta 
Assessment  Consortium.  Especially  in  remote  rural  districts,  the  opportunity  to  leave 
small  towns  to  access  new  ideas  and  research  findings  at  provincial  or  regional 


102 


The  Learning  Mosaic 


conferences  and  establish  lateral  learning  networks  with  educators  in  implementing  them 
was  priceless.  Districts  would  almost  certainly  not  have  embarked  upon  many 
innovations  and  activities  without  AISI. 

2.5  Would  the  values  of  AISI  continue  without  funding? 

Many  participants  from  elementary  schools  said  the  cultures  of  their  schools  had  changed 
and  the  practices  that  came  about  due  to  AISI  were  now  embedded  in  their  schools.  In 
secondary  schools,  AISI  values  were  embedded  in  some  departments  but  others 
conserved  a  transmission  model  of  education  that  did  not  promote  student  engagement. 
Gains  are  being  made  at  the  high  school  level,  but  AISI  project  leaders  indicated  that 
improvements  require  more  careful  modeling  and  support  for  faculty  over  time  of  the 
kind  that  appears  to  be  the  case  in  elementary  schools. 

The  districts  and  cultures  most  likely  to  sustain  AISI  values  in  the  absence  of  continuing 
funding  are  those  that  already  operate  as  complex  and  effective  learning  communities. 
These  districts  have  established  the  organizational  cultures  that  support  teachers' 
continued  introspection,  collaborative  inquiry,  and  adjustment  of  instructional  practices. 
Such  districts  organize  their  leaders  and  not  just  their  teachers  into  PLCs  to  study  data 
and  research  and  to  inquire  into  and  improve  instructional  supports. 

The  Davis  and  Sumara  study  of  contrasting  district  implementation  indicates  that  some 
districts  organize  their  cultures  more  around  foci  such  as  service  and  management  that 
tend  to  concentrate  leadership  centrally  and  administer  projects  vertically  rather  than 
around  learning,  where  both  leadership  and  innovation  are  distributed  more  laterally  and 
bound  together  by  frequent,  complex  interaction.  The  former  types  of  districts  may  find 
it  difficult  to  accomplish  the  learning  goals  they  have  established  for  themselves  because 
they  conflict  with  pre-established  institutional  cultures  that  make  learning  subsidiary  to 
service  or  management.  AISI  values  can  be  piloted  in  such  organizational  cultures,  but 
they  cannot  become  embedded,  and  they  are  unlikely  to  be  sustained  without  funding  and 
also  a  development  of  networking  structures  within  AISI  that  may  stimulate  productive 
disturbance  of  these  existing  district  cultures. 

Some  educators  expressed  anxiety  that  AISI  funding  might  be  folded  into  base  budgeting, 
let  alone  discontinued  entirely,  especially  in  a  time  of  economic  contraction.  They 
worried  that  without  clearly  marked  funding,  the  sorts  of  innovative,  grass-roots  projects 
associated  with  AISI  will  fade  away.  They  feared  that  without  continued  support  for 
AISI  as  an  autonomous  agency,  their  schools  would  not  prosper  from  the  opportunity  to 
mature  into  the  more  complex  learning  systems  and  sources  of  innovation  that  Alberta 
will  need  for  its  students  to  thrive  in  the  future. 

2.6  What  are  the  future  challenges  and  action  points  for  AISI? 

The  presentation  and  discussion  of  overall  findings  of  our  review  points  to  a  number  of 
challenges  for  AISI  and  to  future  points  where  action  might  usefully  be  taken. 
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•  AISI  and  those  who  participate  in  it  first  have  to  revisit  AISFs  goals  -  is  it 
primarily  concerned  with  improvement  by  disseminating  existing  knowledge,  or 
with  innovation  that  promotes  the  creation  of  new  knowledge? 

•  Parent  and  community  engagement,  stronger  connections  to  universities  and  the 
business  community  and  sponsorship  of  cross-district  networking  are  three  areas 
in  which  connectivity  can  be  significantly  improved. 

•  The  assessment  and  accountability  agenda  can  be  furthered  by  reviewing  the  ways 
in  which  student  achievement  data  are  collected  and  tracked  and  connected  to  data 
about  school  characteristics;  by  helping  teachers  to  identify,  design  and  employ 
more  sophisticated  instruments  and  indicators  of  impact  and  progress  that  are 
aligned  with  their  project  goals;  and  by  mitigating  teachers'  feelings  of 
accountability-overload  by  using  measures  in  the  spirit  of  assessment  for  learning 
that  have  direct  diagnostic  and  improvement  value  as  well  as  abandoning  themes 
and  strategies  that  are  no  longer  priorities. 

•  A  more  explicit  and  systemic  theory  of  leadership  development  may  help 
transform  the  district  and  school  cultures  that  serve  as  "hosts"  for  AISI  projects, 
and  lead  to  more  effective  forms  of  distributed  leadership  among  the  wider 
professional  community. 

•  A  clear  and  urgent  strategy  to  reach  and  influence  more  high  schools  will 
significantly  enhance  AISI's  credibility,  as  well  as  begin  to  make  a  dent  in  the 
provincial  dropout  rate. 

•  Last,  in  Cycle  4,  the  attention  being  paid  to  leadership  development  and 
networking,  as  well  as  to  more  flexible  ways  of  administering  timelines  for 
funding  may  begin  to  move  AISI  from  being  a  set  of  discrete  or  clustered  projects 
with  defined  beginnings  and  ends  to  an  integrated,  long-term,  complex  policy 
process  of  improvement  and  innovation  that  forms  a  truly  province-wide  mosaic 
of  learning. 

These  are  the  key  findings  and  challenges  posed  by  our  multiple  perspectives  review.  Our 
final  task  is  to  set  out  some  recommendations  for  further  development  arising  from  them. 
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Chapter  8:  Recommendations 

By  Andy  Hargreaves,  Robert  Crocker, 
Brent  Davis,  Lori  McEwen,  Pasi  Sahlberg, 
Dennis  Shirley  &  Dennis  Sumara 

1.  Introduction 

This  final  chapter  of  our  report  makes  suggestions  and  recommendations  for  the  future  of 
AISI.  These  are  drawn  and  developed  from  the  same  sources  as  our  findings. 

As  we  present  these  recommendations,  we  do  so  from  the  combined  perspectives  of 
appreciative  inquiry  and  critical  friendship  that  have  guided  this  review  and  all  its 
components.  We  have  been  impressed,  at  every  point,  by  the  openness  and  receptiveness 
of  the  School  hnprovement  Branch  (SIB)  team  to  constructive  feedback.  The  absence  of 
defensiveness  is  admirable.  Our  findings  and  recommendations  are  addressed  to  a  culture 
and  community  we  have  found  to  be  characterized  by  the  most  productive  combination  of 
commitment  and  doubt. 

We  note,  for  example,  how  responsive  the  SIB  team  has  been  to  the  discussions  about 
AISI,  its  impact  and  its  future  to  which  AISI  staff,  AISI  partners,  participants  from  AISI 
sites,  as  well  as  critical  friends  such  as  ourselves  contributed  during  the  October  2008 
AISI  Colloquium.  Many  of  the  new  submission  criteria  for  AISI  Cycle  4  are  clearly  a 
direct  result  of  this  organizational  learning,  and  spirit  of  self-critical  inquiry  in  pursuit  of 
professionalism,  improvement  and  enhanced  learning  for  all  students  that  epitomized  that 
Colloquium.  Thus,  many  of  our  recommendations  in  areas  such  as  leadership,  network 
development,  cross-district  collaboration,  and  extended  partnerships  have  already  been 
signaled  by  AISI  leaders  as  key  priorities  for  Cycle  4. 

As  an  advanced  organizer,  our  recommendations  for  future  action  particularly  concern 
the  following  issues: 

1.  develop  improved  ways  of  collecting  and  compihng  provincial  achievement  data 
that  will  make  it  possible  to  trace  the  impact  of  complex  but  distinct  initiatives 
like  AISI; 

2.  create  leadership  and  support  systems  for  teachers  and  administrators  involved  in 
AISI  projects  to  access  existing  data  bases,  request  and  receive  data  analysis 
services,  and  design  their  own  instruments  and  indicators  of  accountability  that 
are  appropriate  to  their  project  goals; 

3.  extend  AISI  project  content  and  processes  towards  greater  involvement  of  parents, 
community  members,  businesses,  universities  and  other  partners; 
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4.  increase  AISI's  attention  to  and  impact  regarding  innovation  and  improvement  in 
high  schools,  with  particular  reference  to  increasing  Alberta's  relatively  low  rates 
of  high  school  completion; 

5.  invest  in  province- wide  networks  that  cut  across  districts,  that  reach  beyond 
annual  conferences  and  that  incorporate  proven  design  principles  of  effective 
network  architectures  that  have  clear,  positive  impacts  on  system-wide  outcomes 
for  students; 

6.  develop  leadership  skill  and  capacity  among  all  principals  and  district-level 
leaders  so  that  the  effectiveness  of  AISI  projects  does  not  suffer  when  existing 
leadership  capacity  in  particular  schools  and  districts  is  not  strong; 

7.  embed  AISI  into  Alberta  Education  as  an  integrated  policy  strategy.  Do  this 
without  diminishing  the  attention,  resources,  advocacy  and  professional 
development  regarding  the  distinctive  approaches  to  professionally  driven,  locally 
adaptable  and  laterally  networked  processes  of  improvement  and  innovation  that 
AISI  has  championed. 

2.  Six  Thematic  Areas 

Our  recommendations  are  grouped  into  six  thematic  areas: 

1 .  Preservation 

2.  Purpose  and  focus 

3.  Impact 

4.  Culture 

5.  Structure  and  funding 

6.  Leadership 

2.1  Preservation 

AISI  should  be  continued  as  a  provincial  priority.  AISI  is  a  unique,  world-leading 
strategy  for  developing  innovation,  and  improving  professional  quality  and  engagement 
in  teaching.  At  the  end  of  its  first  decade,  it  shows  clear  evidence  of  positively  impacting 
teacher  professionalism,  of  creating  and  sustaining  a  professional  culture  of  inquiry  and 
innovation,  and  of  serving  as  a  professionally  inclusive  vehicle  for  delivering  a  number  of 
key  provincial  reform  and  improvement  priorities  such  as  professional  learning 
communities  and  assessment  for  learning.  AISI  does  so  within  a  budget  that  is  a  very 
small  percentage  of  the  overall  provincial  total  for  education,  and  without  any  evidence 
of  prejudice  to  standards  and  achievement  results  on  provincial  achievement  tests  in  a 
context  of  shifting  student  and  community  demographics  that  are  bringing  more 
immigrant  groups  and  second  language  learners  into  the  province. 

Provincial  achievement  tests  and  similar  measures  are  important  instruments  of 
monitoring  traditional  kinds  of  achievement  and  ensuring  public  accountability.  Some 
have  held  out  the  hope  that  AISI  would  have  positive  consequences  for  achievement 
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measured  in  these  terms,  though  because  of  the  Umitations  of  existing  data  sources  and 
collection  procedures,  demonstrating  any  such  connection  has  been  elusive  despite  the 
most  stringent  efforts.  Nevertheless,  AISI  does  not  appear  to  harm  provincial 
achievement  scores  at  a  time  when  it  is  releasing  innovation  and  renewal  into  the 
education  system  of  Alberta. 

Vitally,  AISI  also  addresses  and  fulfils  other  important  goals  within  the  public  education 
system  of  the  province.  In  an  era  following  a  global  economic  crisis,  and  at  a  time  of 
great  volatility  in  energy  prices  and  demands,  economic  diversification  is  an  essential 
public  policy  priority  for  every  jurisdiction,  including  Alberta.  Education  systems 
therefore  need  to  demonstrate  outcomes  beyond  conventionally  tested  basics  in  order  to 
create  flexible  cultures  of  creativity  and  innovation  in  schools  and  society,  and  in  ways 
that  harness  and  heighten  the  capacity  of  high  performing  teams  to  deliver  what  are  now 
being  described  as  inalienable  21^^  Century  skills  for  economic  competitiveness  and 
ecological  survival  -  and  all  within  a  context  of  growing  cultural  diversity  (OECD,  2001; 
Hargreaves,  2003;  Wagner,  2008;  Partnership  for  21''  Century  Skills,  2009).  Students 
must  be  educated  to  join  the  innovative,  high-performing  teams  of  the  future  economy. 
Engaged  and  innovative  teachers  who  model  such  high  performing  teams  and  share  their 
skill  sets  with  their  students  are  the  only  possible  professionals  who  can  deliver  those 
outcomes  for  our  students. 

AISI's  strengths  and  distinctive  contributions  that  are  particularly  worth  preserving  are: 

•  Its  culture  of  creatively  combining  existing  resources,  ideas  and  assets  in  the 
service  of  continuing  and  sometimes  disruptive  innovation.  AISI  is  a  vehicle  for 
enhancing  implementation  of  provincial  priorities,  but  it  is  also  a  funded  and 
favored  niche  for  fostering  innovations  and  innovative  cultures  that  extend,  add  to 
and  sometimes  even  creatively  disrupt  those  priorities.  It  is  unlikely  that  a  diffuse 
and  even  disruptive  culture  of  innovation  could  be  sustained  without  the  targeted 
efforts  of  AISI. 

•  Its  strategy  of  targeted  funding.  This  ensures  that  amid  all  its  other  pressures  and 
short-term  demands,  the  provincial  education  system  will  continue  to  prioritize 
the  innovation  and  professional  engagement  that  is  essential  to  the  province's 
long  term  future.  Given  that  it  is  in  the  nature  of  innovation  that  some  of  it  will 
fail  and  not  all  of  it  will  yield  immediately  measurable  results  that  can  obviously 
be  scaled-up.  Alberta's  politically  stable  environment  is  an  immense  asset  in 
enabling  the  province  to  maintain  its  commitment  to  supporting  educational 
innovation  through  AISI.  In  The  Innovator's  Dilemma,  Clayton  Christensen 
(1997)  shows  that  most  organizations'  initial  innovations  tend  to  become 
normalized  over  time,  inhibiting  other  innovations  in  the  future.  This  is  the 
paradox  of  innovation  and  sustainability.  His  advice,  therefore,  is  that  all  large 
organizations  should  allocate  a  definable  portion  of  budget  to  supporting  future 
innovations,  even  and  especially  when  they  challenge  those  that  have  been 
successfully  been  implemented  in  the  past. 
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•  Its  promotion  and  achievement  of  increased  teacher  professionalism.  High 
performing  education  systems  such  as  that  in  Finland  are  founded  on  being  able  to 
attract,  retain  and  develop  teachers  who  practice  strong  professionalism  together 
in  relationships  of  curricular  and  pedagogical  collaboration  (Hargreaves,  Halasz, 
&  Pont,  2008;  Sahlberg,  2006).  Alberta  already  has  a  strong  and  internationally 
envied  tradition  of  involving  the  teaching  profession  in  significant  system-wide 
changes  such  as  curriculum  developments  and  provincial  achievement  test 
designs.  AISI's  results  point  to  further  and  more  widely  distributed  effects  of 
heightened  teacher  professionalism.  At  the  very  least,  there  is  a  positive 
Hawthorne  effect  of  teachers  valuing  initiatives  that,  in  turn,  value  their  skills, 
judgment  and  involvement.  More  than  this,  AISI  provides  teachers  with  clear 
opportunities  for  making  self-initiated  changes  that  they  especially  value,  rather 
than  implementing  externally  imposed  mandates  that  come  from  afar  (Hargreaves, 
2005).  It  also  creates  many  new  opportunities  and  expectations  for  teacher 
leadership  of  colleagues  and  of  improvement  efforts.  This  is  increasingly  being 
acknowledged  as  essential  to  successful  school  improvement  (Harris,  2008; 
Spillane,  2006). 

•  Its  increasing  commitment  to  networking  across  schools.  Most  teachers  do  not 
perform  consistently  well,  or  learn  how  to  improve,  if  they  teach  entirely  alone. 
This  is  why  there  are  professional  learning  communities.  Most  schools  do  not 
improve  either  unless  they  are  able  to  learn  from  other  schools.  This  is  why  it  is 
important  to  develop  networks.  In  the  past,  networks  have  often  been  open-ended 
and  permissive  and  created  a  lot  of  interaction  and  activity  but  without  much 
impact.  At  the  other  extreme,  in  other  provinces  and  systems,  schools  are 
sometimes  brought  together  in  clusters  to  implement  system  mandates,  but  this 
does  not  induce  innovation  or  increase  professionalism.  The  reports  comprising 
this  review  show  clear  evidence  of  networks  emerging  across  a  number  of  school 
districts  that  increase  the  connectivity  that  is  essential  for  innovation  and 
improvement  in  complex  and  fast-moving  systems.  AISI  has  a  particular  and 
prioritized  network  architecture  for  developing  this  connectivity.  Although  this 
architecture  requires  further  development,  it  is  a  unique  aspect  of  AISI  that  is 
unlikely  to  originate  in  or  be  sustained  by  any  other  aspect  of  provincial  strategy. 

^  •    Its  distinctive  commitment  to  developing  a  widespread  culture  of  teacher-based 
inquiry.  The  commitment  to  teacher-driven  research  and  inquiry  is  not  unique  to 
AISI,  but  the  form  it  takes,  and  the  extent  of  its  reach  are.  Elsewhere,  advocates  of 
professional  learning  communities  support  and  even  institutionalize  school-wide 
analyses  of  data  as  a  basis  for  action,  but  these  inquiry  activities  are  often 
confined  to  and  consumed  by  preoccupations  with  identifying  statistical  gaps  or 
shortfalls  and  designing  rapid  interventions  that  will  yield  short-term  results 
(Datnow,  Park  &  Wohlstetter,  2007;  Dufour  &  Eaker,  1998;  Shirley  & 
Hargreaves,  2006).  Indeed,  some  provinces  and  countries  have  implemented  such 
processes  extensively  as  key  elements  of  their  reform  strategy  (Barber,  2007; 
Fullan,  2006;  Levin,  2008).  AISI  offers  a  different  approach  to  inquiry  that 
engages  in  deeper,  mindful  and  more  sustained  engagements  with  the  nature  of 
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student  learning  and  with  the  changes  that  teachers  need  to  make  in  their  own 
teaching  (Cochran-Smith  &  Lytle,  2009;  MacDonald  &  Shirley,  2009).  This  is 
more  likely  to  lead  to  longer-term  transformations  in  teaching  and  learning  that 
are  more  suited  to  the  development  of  21^^  Century  knowledge  and  skills. 
•    Its  emphasis  on  innovation  and  interaction  with  accountability.  AISI  provides  a 
protected  space  for  innovation  and  incentives  for  increased  professional 
interaction,  combined  with  the  essential  accompaniment  of  internal  and  external 
accountability.  Not  all  AISI  educators  are  satisfied  with  the  expectations  for 
accountability,  and  we  have  some  ensuing  recommendations  for  streamlining 
them  as  well  as  making  them  more  stringent,  but  the  inclusion  of  accountability 
criteria  related  to  reporting  what  was  done,  determining  its  impact,  and  designing 
revised  plans  as  a  consequence,  is  an  indispensible  ingredient  of  an  initiative  like 
AISI.  This  does  not  mean  that  educators  involved  in  AISI  projects  should  feel 
pressed  to  demonstrate  early  or  eventual  success  on  every  occasion  -  the  point  is 
to  demonstrate  and  stimulate  learning,  even  and  often  from  failure  as  much  as 
from  success;  for  this  is  the  essence  of  innovation  and  the  relentless  quest  for 
continuing  improvement. 

2.2  Purpose  and  Focus 

After  ten  years,  and  more  than  three  cycles,  it  is  time  for  AISI  to  clarify,  renew  and 
redefine  its  fundamental  goals.  AISI  is  already  pushing  its  program  priorities  into  new 
areas  such  as  leadership,  parent  and  community  engagement  and  network  development; 
many  of  which  are  not  only  worthwhile  initiatives  and  directions  in  themselves  but  they 
are  also  ones  that  will  help  create  increasing  cohesion  for  AISI  as  a  strategy  of 
knowledge  development  and  dissemination. 

It  is  important  for  AISI  to  establish  clear  priorities.  Sometimes  these  purposes  will  align 
closely  with  and  help  implement  the  strategic  thrusts  of  Alberta  Education  more  widely. 
This  approach  carries  both  benefits  and  risks.  We  note  that  different  cycles  have 
witnessed  a  significant  critical  mass  of  projects  devoted  to  differentiated  instruction, 
professional  learning  communities  and  now,  assessment  for  learning.  In  many  ways,  this 
synchronization  of  AISI  with  wider  strategic  directions  is  admirable  and  to  be 
encouraged.  At  the  same  time,  it  is  evident  that  these  thrusts  and  priorities  typically  have 
packaged  programs  of  external  trainers  and  training  attached  that  can  be  high  in 
consultant  costs,  ephemeral  in  influence  and  evanescent  in  impact,  once  the  trainers  and 
their  albeit  engaging  and  entertaining  workshops  have  passed  by.  There  is  a  concomitant 
risk  that  training  and  trainer-dependent  packaging  on  a  large  scale  can  actually  create  the 
kinds  of  professional  dependency  on  outside  expertise  that  AISI  in  many  ways  is 
designed  to  counteract  with  its  emphasis  on  school-initiated  innovation. 

These  issues  of  synergy  raise  questions  about  whether  AISI  is  now  so  institutionally 
embedded  in  the  system  as  a  way  of  making  change  that  it  should  no  longer  be  treated  as 
a  separate  initiative.  At  the  same  time,  data  that  show  early  signs  of  some  tailing  off  in 
teacher  satisfaction  levels  and  other  outcome  measures  suggest  a  need  for  other 
approaches  within  the  AISI  design  to  keeping  innovation  going  and  pushing  new 
initiatives  forward.  And  it  is  clear  that  while  some  aspects  of  AISI  are  indeed  embedded 
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into  the  wider  policy  and  change  process,  others,  such  as  its  approach  to  networking  and 
interconnectivity  among  schools  have  not  yet  been  pushed  or  embedded  far  enough. 

Finally,  in  considering  our  recommendations  in  the  realm  of  purpose  and  focus,  we  are 
aware  that  the  impact  of  AISI  seems  weakest  in  the  high  school  sector.  With  high  dropout 
rates  being  a  significant  concern  within  the  province,  it  is  important  to  find  a  way  of 
increasing  AISFs  energy  and  impact  within  high  schools  as  a  way  to  leverage  innovation 
in  this  traditionally  hard-to-change  domain.  Too  few  AISI  projects  have  been  focused  on 
improving  high  school  completion  rates  -  a  highly  targeted  area  that  would  also  be  easy 
to  measure. 

Taking  into  account  our  findings  on  AISI  and  its  impact,  we  therefore  advance  the 
following  recommendations  about  purpose  and  focus: 

•  Rename  and  redirect  AISI  as  a  process  or  a  network;  not  an  initiative.  Initiatives 
in  excess  eventually  become  professionally  discredited  as  patterns  of  innovation 
and  improvement.  After  a  decade  of  operation,  it  is  also  stretching  the  argument 
to  call  anything  an  initiative  any  more.  The  designation  of  AISI  as  a  network  or 
process  will  be  a  signal  of  AISFs  robust  embedding,  rather  than  conditional  and 
marginal  presence  within  an  ongoing  approach  to  innovation  and  improvement  in 
Alberta  Education. 

•  Re  conceptualize  AISI  as  a  bridge  to  21^^  century  learning.  AISFs  distinctive  asset 
and  approach  is  not  in  delivering  better  results  in  existing  and  more  conventional 
approaches  to  learning  and  achievement.  It  is  in  developing,  deepening  and 
delivering  the  new  kinds  of  learning  that  are  essential  for  the  province's 
competitiveness  in  a  future  knowledge  economy  (OECD  2001,  2008)  and  also  for 
its  contribution  to  combating  and  adapting  to  the  imminent  threat  of  climate 
change  (Giddens,  2009),  through  embracing  the  international  advocacy  for 
education  for  sustainable  living  (Senge,  forthcoming). 

•  Refocus  AISI  around  innovation  and  renewal  as  well  as  improvement.  It  should 
more  strongly  and  explicitly  promote  innovation  and  creation  of  new  knowledge; 
not  only  dissemination  and  diffusion  of  existing  knowledge  that  is  embedded  in 
provincial  priorities,  or  in  the  packaged  programs  of  externally  hired  consultants 
and  trainers.  Our  evidence  indicates  that  AISI  projects  already  sometimes  create 
new  knowledge.  They  innovate,  inquire  and  initiate  as  well  as  implement  and 
disseminate.  This  does  not  mean  that  all  new  knowledge  should  start  with  the 
school.  It  is  not  the  origin  of  knowledge  that  matters  but  the  way  it  is  processed. 
The  strongest  and  best-led  school  cultures  are  able  and  eager  to  take  knowledge 
from  many  sources  (including  central  programs  and  external  trainers)  and  make  it 
their  own  by  connecting  it,  combining  it,  reworking  it,  inquiring  into  it  and 
adapting  it  to  their  own  needs  and  circumstances  (Abrahamson,  2004;  Hargreaves, 
2005).  Understanding  and  extending  this  capacity  for  creating  new  local  and 
generahzed  knowledge  of  learning  is  an  immense  opportunity  for  AISI. 
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•  Concentrate  disproportionate  ejforts  and  resources  on  innovation  and 
improvement  in  high  schools.  AISI  currently  seems  to  make  the  weakest  inroads 
into  high  schools.  High  schools  are  the  leviathans  of  school  reform  and 
improvement  -  large,  content-driven,  constrained  by  examinations  and  framed  by 
university  qualification  requirements  (Goodson,  1994).  The  strongest  leverage  for 
initial  change  is  often  in  smaller  subjects,  among  younger  teachers,  with  diverse 
populations,  in  traditionally  more  marginalized  areas  like  counseling  and  special 
education,  or  within  lower  grade  levels  (Skerrett  &  Hargreaves,  2008).  The 
challenge  is  then  how  to  spread  innovation  throughout  the  higher  status  areas  of 
the  high  school  community.  Achieving  greater  success  with  AISI  in  high  schools 
is  unlikely  to  come  about  just  by  greater  exhortation.  Focused  effort  and  targeted 
funding  may  be  required,  with  perhaps  disproportionate  and  protected  resource 
allocations  being  directed  towards  high  schools  until  change  momentum  has  been 
established.  Promising  starting  points  might  be  in  the  existing  commitment  to 
improving  high  school  retention;  developing  curriculum  innovation  for  21"^^ 
century  skills  across  the  curriculum  or  within  mainstream  subjects  such  as 
mathematics  and  science;  and  developing  innovation  and  connectivity  in  the  years 
preceding  and  following  student  transition  to  high  school. 

•  Increase  priority  for  proposal  and  actions  that  plan  to  increase  parent  and 
community  engagement  in  their  children's  education.  The  First  Way  of  change 
was  characterized  by  passive  trust  where  parents  unconditionally  trusted 
professionals  with  their  children.  A  period  of  active  mistrust  then  set  in  as  the 
public  looked  to  external  accountability  instruments  to  guarantee  commitment  and 
quality.  In  the  Third  Way,  progress  in  measured  results  secures  public  confidence 
in  the  education  profession.  The  Fourth  Way  moves  on  from  this  position  and 
develops  active  trust  between  professionals,  parents  and  community  members 
working  side  by  side.  This  approach  is  about  developing  and  organizing 
communities  to  be  stronger  advocates  for  and  supporters  of  their  children,  not  just 
about  delivering  government  services  to  communities  in  need.  AISI  can  make  a 
significant  mark  by  promoting  more  open  kinds  of  teacher  professionalism  in 
which  the  boundaries  between  teachers  and  diverse  communities  weaken,  and 
patterns  of  mutual  influence,  trust  and  respect  grow  actively  between  them. 

•  In  summary,  highlight  and  broadcast  AISVs  prime  purposes  as  ones  that 
encompass  improvement,  innovation,  and  professional  engagement  to  develop 
and  demonstrate  21^'  Century  learning.  This  refinement  of  purpose  and 
direction,  we  believe,  will  infuse  new  energy  into  AISI  and  the  Alberta  teaching 
profession,  and  maintain  momentum  at  a  time  of  dramatically  shifting  educational 
and  economic  needs. 

2.3  Impact 

Whatever  the  goals  of  AISI  or  any  other  educational  change  strategy,  it  is  crucial  to  be 
able  to  determine  what  impact  the  strategy  is  making.  Impact  assessment  is  essential  for 
establishing  external  accountability,  public  confidence  and  justification  of  resource 
allocation  in  relation  to  other  priorities  on  the  one  hand;  and  as  part  of  the  recurring  cycle 
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of  improvement  across  the  strategy  and  within  every  district  and  school  on  the  other. 
Without  impact  assessment,  confidence  wanes,  resources  become  hard  to  justify,  and 
there  is  no  evidence  or  feedback  to  guide  or  prompt  efforts  to  improve  or  refine  chosen 
strategies. 

Our  review  indicates  that  educators  are  sometimes  critical  and  even  resentful  of  existing 
accountability  mechanisms.  Required  paperwork  at  the  end  of  each  cycle  seems  excessive 
in  its  call  for  narrative  accounts  of  what  actions  were  taken  within  projects  and  what 
consequences  ensued.  This  sometimes  makes  AISI  feel  like  an  imposition  rather  than  an 
opportunity.  Many  of  the  quantitative  measures  chosen  to  document  impact  seem  to  be 
selected  because  of  their  easy  availability  rather  than  their  usefulness  in  evaluating 
particular  projects.  Provincial  achievement  test  data  are  easily  accessed,  for  example,  but 
not  necessarily  the  most  useful  as  measures  of  efforts  to  develop  new  kinds  of  learning. 
Satisfaction  surveys  similarly  often  seem  bland,  weak,  and  too  easily  pulled  down  from 
the  provincial  menu  of  instruments  rather  than  acting  as  fine-tuned  instruments  to  assess 
precise  forms  of  impact.  There  can  also  be  a  culture  of  implied  expectation  to  exaggerate 
or  fabricate  success  when,  in  the  case,  of  innovation,  it  might  be  more  useful  to 
demonstrate  what  has  been  learned  and  what  actions  might  be  taken  as  a  result.  Finally, 
the  existing  program  structure  of  three-year  cycles  can  make  accountability  requirements 
feel  excessive,  repetitive  and  overwhelming. 

In  reviewing  our  evidence,  the  conclusion  we  reach  about  impact  and  accountability 
requirements  is  that  they  should  be  streamlined,  made  more  stringent,  be  more  embedded 
and  actively  utilized  within  each  project's  process  of  diagnosis  and  inquiry,  and  be  the 
subject  of  expert  and  ongoing  technical  support  that  is  differentiated  according  to  levels 
of  need  and  existing  capacity  in  each  project.  Specifically,  this  leads  us  to  make  the 
following  recommendations: 

•  Streamline  external  accountability  requirements  by  reducing  the  length  and 
frequency  of  external  narrative  reporting.  The  exceptions  would  be  where  there 
was  a  clear  purpose  for  the  reporting  other  than  accountability  per  se.  Thus,  some 
schools  or  districts  might  produce  lengthy  reports  on  a  sample  basis,  if  the  schools 
were  to  be  visited  by  a  review  team  at  a  later  point  to  engage  in  dialogue  with 
them  about  the  future  of  their  project.  Other  reports  might  be  extended  in 
exemplary  cases  for  the  purposes  of  publication  and  celebration  of  successful,  or 
especially  innovative  practice. 

•  Continue  and  extend  systematic  research  into  the  impact  of  AISI  as  a  system.  AISI 
already  has  an  impressive  record  of  cycle-by-cycle  reviews,  quantitative  and 
qualitative  evaluations,  project  self-reports,  and  this  multiple  perspectives  review. 
We  encourage  AISI  to  continue  this  work  and  also  to  go  further  -  especially  in  the 
quantitative  domain,  by  developing  and  utihzing  other  objective  and  robust 
indicators  of  impact  than  the  existing  measures  which  are  readily  accessible  but 
designed  for  other  purposes  and  also,  sometimes,  other  kinds  of  learning  and 
learning  outcomes.  Such  measures  should  include: 
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o    Robust  measures  of  student  learning  and  impact  on  student  learning  in  more 
creative  and  applied  areas  of  21^^  century  skills  such  as  those  measured  by  the 
international  PISA  tests,  those  that  ask  students  and  teachers  what  ways  of 
learning  they  value  most  and  how  often  they  experience  them  etc. 

o    Adoption  of  existing  scales  related  to  stages  of  implementation  and  levels  of 
use  (e.g.,  Concerns-based  Adoption  Model)  in  order  to  determine  how 
successfully  initiatives  have  been  implemented,  whether  these  have  reached 
only  levels  of  awareness  through  external  training  strategies  or  have  been 
integrated  into  teachers'  practices,  and  whether  initiatives  have  spread  beyond 
a  few  early-adopters  to  teachers  whose  approaches  to  change  are  more 
cautious  or  even  resistant. 

•  Establish  an  AISI  Institute  of  Data  (AID).  This  will  comprise  a  systemic  support 
service  that  enables  teachers  and  researchers  to  make  more  efficient  and  effective 
use  of  existing  data,  to  be  aware  of  and  access  the  instruments  that  are  most 
appropriate  for  their  projects,  and  to  develop  their  own  instruments  as  necessary. 
AID  would  collect,  compile  and  compute  data  related  to  individual  student 
numbers  more  swiftly  and  efficiently  than  teachers  themselves  -  thereby 
significantly  streamlining  the  accountability  and  inquiry  process.  As  in  the  long- 
estabhshed  Manitoba  School  Improvement  Project,  AID  could  also  advise 
teachers  and  schools  in  provincial  and  regional  conferences,  as  well  as  district  by 
district,  about  how  to  select,  develop  and  utilize  instruments  that  enhance  the 
process  of  inquiry  as  an  integral  part  of  continuous  improvement.  This  local 
support  function  of  AID  might  be  most  effective  if  the  structure  of  AISI  cycles 
and  the  associated  proposal  process  were  redefined  (see  below). 

•  Initiate  a  project  review  process  on  a  basis  of  sampling,  invitation  and/or  need. 
An  AlSI-wide  continuous  systemic  review  process  might  be  enhanced  further  if  it 
was  not  itself  tied  into  three-year  cycles,  evaluating  projects  that  had  finished, 
once  the  work  was  over,  but  if  it  also  followed  the  principles  of  assessment  for 
learning  and  became  an  integral  part  of  the  improvement  and  innovation  process 
itself.  Thus,  a  central  review  team  established  by  AISI  might  undertake  sample 
reviews  of  projects-in-action,  connecting  an  extended  internal  review  and 
narrative  by  the  project  team,  with  a  2-3  day  site  visit  and  external  review  by  the 
Central  AISI  team.  This  would  contribute  to  monitoring  of  quality,  and  to 
assessing  the  ongoing  needs  for  both  celebration  and  support  throughout  the 
initiative.  Sites  for  review  may  be  selected  by  random  sample,  by  invitation  from 
the  sites,  by  indications  of  the  opportunity  to  record  instances  of  exceptional 
exemplary  practice,  or  by  need  for  assistance  when  schools  or  districts  appear  to 
be  struggling.  This  leads  to  the  final  recommendation  of  this  section. 

•  Clearly  and  transparently  identify  sites  that  have  achieved  different  levels  of 
development,  impact  or  success  in  terms  of  robust  and  diverse  outcome  measures. 
Without  such  a  clear  and  transparent  system,  successes  cannot  be  celebrated,  nor 
can  weaknesses  or  difficulties  be  pinpointed  and  assistance  then  offered  or 
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assigned  accordingly.  Highly  successful  networks  elsewhere  that  bring  about 
systemic  improvement  (as  well  as  individual  examples  of  innovation)  are 
prepared  to  identify  and  acknowledge  different  levels  of  success  in  terms  of 
impact  (Hargreaves  &  Shirley,  2009).  Only  in  a  transparent  system  of 
differentiated  degrees  of  success  can  schools  offer  and  seek  assistance  from  peers 
who  are  addressing  similar  issues  to  themselves,  and  only  in  such  circumstances 
can  there  be  a  clear  and  accountable  basis  for  providing  external  support.  This 
presents  a  significant  challenge  to  some  existing  cultures  of  teaching  and 
professionalism  that  are  articulated  by  a  particular  kind  of  equity  ethic  -  so  we 
return  to  this  recommendation  later. 

2.4  Culture 

AISI  has  strengthened  and  extended  a  professional  culture  of  teaching  in  Alberta  that  is 
already  distinctive.  It  has  valued  teachers  as  active  agents  of  change,  not  merely  passive 
implementers  of  others'  changes.  It  has  promoted  innovation,  improvement  and  inquiry. 
It  has  validated  the  power  of  discretionary  and  reflective  judgment  as  a  fundamental 
aspect  of  professionalism.  AISI  seems  to  have  exerted  a  positive  Hawthorne  effect,  with 
measures  of  teacher  growth  being  directly  correlated  with  AISI's  development  as  a  result 
of  the  additional  motivation  teachers  get  from  having  their  own  professional  efforts 
recognized,  rewarded  and  resourced. 

AISI  is  the  product  of  collaboration  among  many  partners  and  its  design  promotes 
collaboration  within  and  between  schools  as  a  basic  principle  of  professionalism  and 
change.  Teacher  collaboration  and  growth  within  schools  has  already  resulted  from  AISI, 
partly  through  the  attention  to  developing  professional  learning  communities. 
Collaboration  across  schools  has  also  become  more  extensive  after  the  Venus-like, 
individual  school  initiative  phase  of  Cycle  1.  The  further  development  and  continued 
success  of  AISI  depends  at  this  point  on  extending  and  deepening  collaboration  and 
partnership  in  three  ways:  widening  the  range  of  collaborative  partnerships;  forging 
stronger  connections  and  networks  among  schools  across  districts;  giving  collaborative 
work  a  more  explicitly  critical  edge  and  instigating  collaborative  relationships  of  support 
and  assistance  between  sites  that  are  unequal  in  levels  of  capacity  for  change,  or  in  stages 
reached  in  implementation.  Three  recommendations  follow: 

•    First,  widen  AISI  project  partnerships.  Widening  partnerships  extends  both 
commitment  and  capacity.  Partnering  with  parent  and  community  organizations 
develops  a  more  open  professionalism  and  involves  many  agents  who  can  actively 
contribute  to  the  development  and  success  of  individual  students.  Responsible 
business  partnerships,  where  businesses  give  time  and  service  to  educational 
projects  as  part  of  their  role  as  corporate  citizens  rather  than  in  search  for  direct, 
short-term  returns  on  investment,  builds  capacity  for  improvement,  develops  the 
profile  and  credibility  of  AISI  within  the  business  community,  and  gives  positive 
energy  to  developing  the       century  skills  that  are  essential  to  the  changing 
economy.  Last,  while  many  universities  have  developed  ad  hoc  partnerships  with 
AISI  and  many  include  AISI  links  on  their  website,  there  is  significantly  greater 
opportunity  to  access  the  research  and  inquiry  resources  and  capacity  of 
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universities  by  forming  structured  research  partnerships,  government-university 
research  and  development  collaborations,  and  collaborative  bidding  processes  for 
Federal  grants  linked  to  AISI  through  agencies  such  as  the  Social  Sciences  and 
Humanities  Research  Council.  Enhanced  partnerships  with  universities  will  also 
strengthen  the  utility  of  university  research  for  professional  practice  in  schools,  as 
well  as  cultivating  greater  respect,  understanding  of  and  learning  from 
professional  practice  by  university  research  faculty. 

•    Second,  provide  incentives  for  network  development  across  school  districts.  Since 
Cycle  1  of  AISI,  networking  has  rightly  become  a  progressively  important  priority 
within  each  successive  cycle.  It  is  one  of  the  approved  directions  for  AISI  in 
Cycle  4.  Networks  help  educators  and  their  schools  learn  from  and  improve  with 
one  another.  They  break  down  the  insularity,  privacy  and  even  secrecy  of 
individual  schools  and  their  districts.  They  provide  a  means  for  strong  and 
innovative  practice  to  be  diffused  and  for  successes  to  be  scaled  up  across  the 
system.  They  also  offer  opportunities  for  institutions  and  individuals  with  greater 
capacity  in  particular  areas  to  assist  professional  peers  who  are  less  well  equipped 
or  who  have  not  progressed  as  far.  Networks  are  increasing  in  strength  and 
effectiveness  in  many  districts,  but  not  in  all  of  them.  They  are  stronger  among 
teachers,  very  often,  than  among  administrators  -  though  a  few  districts  have 
established  professional  learning  communities  across  schools  among 
administrators.  Apart  from  annual  AISI  conferences  and  occasional  workshops 
though,  network  development  across  districts  is  weak.  This  limits  progress  in 
developing  the  further  learning,  improvement  and  cohesion  that  is  essential  to 
AISI's  success. 

Connections  among  districts  typically  have  been  coordinated  centrally  in  relation  to 
government  implementation  priorities.  Inter-district  coordination  also  occurs  through 
conferences,  meetings  and  other  kinds  of  representation  among  district-level  leaders. 
These  can  often  be  effective  mechanisms  for  delivering  centrally  driven  initiatives  and 
strategies.  They  are  ineffective  structures,  however,  for  coordinating  and  connecting 
school-initiated  strategies  of  the  kind  developed  within  AISI.  With  no  effort  at  inter- 
district  coordination,  school  initiatives  become  disconnected  and  diffuse  as  in  Cycle  1.  If 
all  the  direction  comes  from  the  Centre,  then  AISI  simply  turns  into  another  arm  of 
government  implementation  -  in  relation  to  already  established  priorities,  training 
packages  and  the  like;  lessening  the  likelihood  of  both  innovation  and  sustainability,  and 
contrary  to  the  intended  culture  and  purposes  of  AISI. 

There  is  a  powerful  need,  therefore,  to  encourage  and  create  incentives  for  district-to- 
district  and  cross-school/cross-district  networking  sometimes  with  and  sometimes 
independently  of  district  leadership  control  as  a  way  to  build  capacity,  accelerate 
diffusion  and  enhance  organizational  learning.  This  is  an  important  priority  everywhere 
but  may  be  most  needed  in  those  districts  where  school-based  AISI  projects  make  the 
least  progress  because  of  excessively  weak  or  overly  controlling  district  leadership.  In 
these  cases,  AISI  networking  could  actually  create  productive,  disruptive  innovation 
within  districts  of  the  kind  that  Christensen  (1997)  describes.  Drawing  on  experiences 
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with  successful  networks  and  partnerships  elsewhere  (e.g.,  Hargreaves  &  Shirley,  2009), 
inducements  and  incentives  might  lead  AISI  staff  to: 

o    Target  funding  allocations  within  the  overall  AISI  budget  for  cross-district 
collaborative  projects 

o    Require  plans  for  networking  of  results  and  activities  across  districts  within  all 
AISI  submissions 

o    Further  review  AISI's  website  design  to  increase  attention  to  cross-district 
knowledge-sharing  -  although  the  Clearinghouse  is  already  rich  in 
information 

o    Structure  conference  activities  so  they  do  not  only  promote  showcasing  of 
exemplary  efforts,  but  also  expect  schools  to  seek  out  and  interact  with 
partners,  and  set  norms  for  transparent  displays  of  participation  and  results 
among  all  AISI  projects 

o    Provide  a  protected  travel  budget  for  teacher  inter-visitations  across  the 
province 

o    Designate  funding  for  a  leader  of  cross-district  networks  and  other 
partnerships 

•    Third,  create  processes  and  protocols  for  critical  discernment  of  professional 
practice.  The  culture  of  teaching  is  sometimes  reluctant  to  publicly  accept  or 
acknowledge  different  levels  of  expertise  or  degrees  of  success  in  professional 
practice.  While  it  is  immensely  important  to  share  and  celebrate  successes  and 
communicate  examples  of  interesting  and  innovative  practice,  the  culture  of 
teaching,  like  the  culture  of  self-esteem,  can  sometimes  produce  an  over- 
celebration  of  all  practice  that  admits  the  existence  of  neither  exemplary  success 
nor  of  problems  and  shortcomings.  Learning  and  improvement  involve  learning 
from  mistakes,  and  working  with  mentors  whose  performance  is,  initially, 
superior  or  advanced.  This  does  not  mean  that  stronger  or  weaker  schools  or 
districts  perform  better  or  worse  on  everything  -  but  it  does  mean  being  able  to 
acknowledge  and  identify  different  levels  as  well  as  types  of  professional  practice. 
Connecting  schools  that  are  operating  at  different  levels  with  similar  types  of 
students  can,  as  other  provinces  and  countries  have  demonstrated,  lead  to 
narrowing  of  achievement  and  learning  gaps  (Fullan,  2006;  Hopkins,  2007).  This 
connectivity  is  vital  not  only  when  implementing  central  reforms  such  as  literacy 
strategies  however,  but  also  among  schools  attempting  similar  innovations  or 
initiatives,  or  with  variable  capacity  in  leadership  or  networking  capabilities. 
Deepening  the  critical,  reflective  dialogue  among  AISI  sites  can  be  achieved  by  a 
number  of  measures,  including: 
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o    Self  evaluation  and  external  evaluation  processes  that  engage  insider  dialogue 
with  outsider  perspectives 

o    Deployment  of  specific  protocols  for  professional  interaction  such  as  that 
used  by  critical  friends  networks,  that  promote  feedback  that  is  constructive, 
affirming  and  also  critical  in  nature. 

o    Use  of  more  robust  internal  evaluation  instruments  such  as  those  that  assess 
impact  on  student  learning,  and  stages  of  implementation 

o    Deliberate  and  transparent  partnering  ofAISI  sites  that  have  different  levels  of 
expertise,  capacity  or  progress  in  implementation 

o    Developing  strong  norms  of  specific  public  self-criticism  rather  than  more 
generalized  kinds  of  modesty  and  humility  in  the  most  exemplary,  successful 
cases  so  they  model  the  value  and  necessity  of  learning  from  mistakes  to  their 
peers 

2.5  Structure  and  Funding 

Practices  are  embedded  in  cultures  that  value  particular  kinds  of  interaction  among 
members  of  a  community.  Structures  of  roles,  resources,  time  and  space  both  create  and 
constrain  opportunities  for  these  kinds  of  interaction.  By  intent  or  by  accident,  they 
permit  and  push  some  practices  and  also  prohibit  others.  AISI  has  several  key  structures 
that  are  linked  to  its  purposes.  These  include  a  formula  for  distributing  resources  across 
geographical  space;  funding  cycles  that  orchestrate  project  activities  over  time;  proposal 
procedures  that  signal  criteria  of  acceptance  and  rejection;  and  design  features  such  as 
province- wide  conferences  that  promote  or  proscribe  patterns  of  interaction  between  AISI 
schools  and  districts.  In  line  with  our  observations  about  the  shifting  purposes  and 
priorities  of  AISI,  with  the  need  to  have  a  sustainable  as  well  as  successful  strategy,  with 
the  need  to  maintain  and  renew  momentum,  and  with  the  need  for  a  more  challenging 
culture  of  professional  interaction  and  evaluation,  we  make  several  recommendations  in 
relation  to  reviewing,  revising  and  renewing  the  structure  and  funding  of  AISI. 

•    Redesign  a  more  flexible  funding  and  proposal  cycle.  Three-year  AISI  cycles  may 
be  too  long  or  too  short  -  it  depends  on  the  purposes.  Some  sites  may  want  to 
experiment  with  a  small-scale  innovation  in  literacy  for  boys.  Others  may  want  to 
transform  their  entire  high  school  structure.  The  time  scales  for  design, 
implementation  and  possible  success  in  each  of  these  cases  are  profoundly 
different.  If  everything  is  subjected  to  an  identical  timeline  irrespective  of  need,  it 
converts  a  cycle  into  a  treadmill  -  as  a  number  of  participants  more  than  hinted  at 
in  their  remarks.  Auto  assembly  lines  in  the  2V^  century  economy  now  have  to  be 
flexible,  not  standardized.  Public  sector  organizations  need  to  follow  the  same 
customized  path  towards  increased  and  enhanced  flexibility.  AISI  already  leads 
the  way  in  this  respect  in  supporting  projects  of  1,  2  or  3  years'  duration.  In 
addition,  it  encourages  submissions  that  build  on  and  deepen  past  projects.  This 
flexibility  can  be  extended  by  encouraging  proposals  for  longer  term  projects  at 
the  outset,  the  first  stages  of  which  are  the  target  of  requested  funding.  There  is 
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also  evidence  that  some  educators  regard  the  three-year  cycles  as  requirements 
and  there  perhaps  needs  to  be  clearer  and  stronger  communication  about  the 
options  available. 

Develop  funding  formulae  that  are  demonstrably  tied  to  and  differentiated  by 
need  in  terms  of  factors  such  as  the  difficulty  and  complexity  of  the  project  being 
undertaken,  the  degree  of  innovativeness  suggested  by  the  project,  the  amount  and 
depth  of  collaborative  engagement  with  other  partners,  the  importance  of  the 
priority  within  AISI  and  the  province  (for  instance,  that  of  high  school  reform) 
and  the  size  of  the  unit  undertaking  it.  This  raises  the  possibility  of  differentiating 
and  targeting  funding  into  different  "pots"  that  each  become  the  subject  of 
different  groups  of  proposals  and  bids.  Such  a  process  would  customize  resource 
allocation  more  closely  to  need,  release  it  on  flexible  time  schedules  driven  by  the 
nature  and  need  of  the  project  and  the  characteristics  of  the  site  bidding  for  it,  and 
sharpen  the  thinking  and  planning  of  proposers  in  relation  to  the  focus  and 
direction  of  their  projects.  This  would  represent  a  step  away  from  the  current  use 
of  per-capita  funding  which  is  less  adaptable  and  responsive  to  local  need. 

Provide  a  centralized  resource  for  data  collection  and  management,  and 
reallocate  some  AISI  funding  for  this  from  individual  site  proposals  to  the 
previously  mentioned  agency  we  tentatively  named  AID  (AISI  Institute  of  Data), 
in  order  to  streamline  the  processes  of  accountability  and  site-based  inquiry. 

Define  clear  guidelines  and  criteria  regarding  proscribed  uses  of  AISI  funding  - 
for  instance,  for  proposals  that  use  AISI  resources  to  replace  professional 
development  funding  in  a  district;  for  proposals  that  have  no  plans  for 
sustainability;  or  for  proposals  that  establish  physical  or  technological 
infrastructure  rather  than  promoting  and  enhancing  new  and  improved  practices  of 
teaching  and  learning. 

Redesign  the  proposal  process  from  one  based  on  selection  to  one  that  explicitly 
promotes  learning  and  development  so  that  the  most  common  outcome  is  not  one 
of  acceptance  or  rejection  but  of  resubmission  with  the  assistance  of  a  technical 
support  team  of  AISI  advisors.  This  team  can  provide  support  on  such  matters  as 
how  to  collect  and  compile  impact  data,  what  factors  to  consider  in  relation  to 
sustainability,  and  how  to  use  trainers  effectively  so  that  sites  interact  intelligently 
with  the  models  advocated  by  trainers,  rather  than  implementing  external 
initiatives  uncritically.  The  proposal  process  itself  thereby  becomes  a  distinctive 
feature  of  AISFs  developmental  design  -  fostering  learning  and  connectivity  even 
at  the  point  of  application. 

Extend  or  reallocate  a  proportion  of  AISFs  resources  to  expand  and  deepen  the 
networks  and  partnerships  that  are  essential  to  its  further  development  and 
impact.  This  might  include  increasing  the  number  of  conferences  that  bring  AISI 
sites  together,  and  doing  so  around  clear  principles  of  transparency  of 
participation  and  results.  It  might  also  include  funding  allocations  for  inter- 
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visitations  across  districts,  especially  where  there  are  clear  gaps  of  expertise  and 
capacity  between  different  sites.  Specific  additional  funding  for  involving  other 
partners  such  as  university  researchers  in  the  development  and  evaluation  of  AISI 
innovations  also  offers  a  useful  way  forward. 

•  Launch  an  international  conference  and  think  tank  bringing  together  experts  with 
practical  experience  of  leading  school  networks  of  innovation  and  improvement 
that  have  clear  connections  to  results.  This  expertise  exists  in  a  number  of 
countries  including  Finland,  Australia,  Singapore  and  England.  Drawing  these 
groups  together  would  help  AISI  redesign  its  architecture  as  an  effective 
province-wide  network. 

2.6  Leadership 

There  is  rarely  lasting  change  without  leadership.  AISI  creates  significant  opportunities 
for  increased  teacher  leadership  and  for  further  career  development  beyond  but  not 
necessarily  instead  of  the  classroom.  AISI,  in  other  words,  increased  leadership  density  in 
schools  and  school  districts  (Sergiovanni,  1984).  Some  AISI  projects  rely  heavily  on 
teachers  promoted  into  coordinator  roles  and  this  raises  the  question  but  also  the 
opportunity  of  developing  internal  leadership  capacity  behind  them  in  their  schools  when 
they  move  into  the  district  office.  Other  districts  develop  more  creative  uses  of  teacher 
leadership  by  buying  proportions  of  time  of  multiple  teachers  so  they  can  also  experience 
leadership  with  and  of  their  colleagues  without  abandoning  their  classroom  roles  and 
leadership  of  students. 

At  the  same  time,  success  in  AISI  projects  seems  to  depend  strongly  on  the  effectiveness 
of  principal  and  superintendent  level  leadership  within  the  district.  Some  districts  and 
their  initiatives  benefited  from  outstanding  leadership  of  more  than  one  kind.  The  ones 
that  appeared  to  struggle  or  falter  had  high-level  leadership  that  was  either  weak, 
excessively  controlling  and  inflexible,  or  isolated  from  other  schools  and  districts.  The 
impact  of  AISI  projects  depended  strongly  on  prior  leadership  capacity  in  being  able  to 
create  a  sense  of  direction  and  purpose  and  unleash  the  innovation  and  connectivity 
among  professionals  across  the  system. 

AISI  is  already  committed  to  developing  leadership  capacity  as  part  of  its  Cycle  4 
priorities.  It  largely  does  so  by  advocating  increased  commitment  to  shared  leadership. 
We  agree  with  the  importance  of  this  emphasis  but  unless  it  is  unpacked  a  little  it  could 
be  misleading.  The  paradox  of  leadership  appears  to  be  that  effective  shared  or 
distributed  leadership  does  not  only  call  for  more  teacher  leadership,  but  also  requires 
sophisticated  levels  of  inner  strength,  courage  and  confidence  among  high  level  leaders. 
In  closing  our  report,  we  therefore  conclude  with  some  recommendations  regarding 
leadership  development. 

•  Make  leadership  development  a  clear  AISI  project  priority  and  desired  outcome, 
not  an  assumed  precondition  of  success.  This  means  sharpening  the  leadership 
emphasis  even  further  in  Cycle  4. 
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•  Continue  to  affirm  and  expand  the  role  and  density  of  teacher  leadership  in  AISI 
schools,  balancing  the  needs  of  district  coordination  with  the  continuation  of 
leadership  capacity  building  within  schools. 

•  Provide  specific  training  and  support  for  principals  and  district  level  leader's,  in 
conjunction  with  their  professional  associations,  in  relation  to  network  leadership 
and  the  development  of  shared  responsibility  for  change. 

•  Promote  focused  interaction  and  networking  among  and  across  school  leaders 
and  district  leaders,  that  are  characterized  by  mutual  support,  candid  discussion, 
honest  recognition  of  differences  in  degrees  of  success  and  implementation,  clear 
protocols  that  promote  critical  dialogue,  and  open  professional  interaction  in  a 
culture  of  collaboration,  inquiry  and  commitment  to  improvement. 

3.  Conclusion 

AISI  is  an  impressive  change  strategy  that  is  perhaps  without  parallel  in  the  world  today. 
It  contributes  to  teacher  development  and  educational  change  in  a  manner  that  is  stable, 
steady,  and  credible  among  the  educators  it  most  seeks  to  impact.  AISI  leadership  is 
transparent,  responsive,  and  trustworthy. 

AISI  has  built  a  solid  foundation  to  further  evolve  and  address  some  of  the  most 
tenacious  problems  in  educational  change  today.  In  the  years  ahead,  AISI  leaders  should 
build  upon  their  many  accomplishments  and  expand  the  most  important  themes  and 
strategies  of  AISI  into  new  arenas.  AISI  should  further  promote  learning  across  district 
lines  and  should  increase  parents  and  community  engagement  in  schools.  More 
concerted  efforts  and  sustained  support  need  to  be  provided  to  high  schools  to  engage 
students  and  to  transform  learning.  In  general,  more  flexible  and  also  more  targeted 
approaches  to  funding  and  funding  cycles  may  help  achieve  these  goals. 

AISI  is  already  promoting  some  of  these  changes  in  its  new  cycle  of  projects.  It  is 
imperative  that  AISI  act  decisively  and  boldly  in  leading  the  changes.  AISI  has  a  unique 
change  architecture.  It  treats  the  learning  of  students,  teachers  and  organizations  not  as  a 
line,  or  even  a  circle,  but  as  a  complex,  interlocking  mosaic.  AISI  is  a  complex  model  of 
improvement  and  innovation  and  also  a  transparent  and  participatory  one.  This  is  why  it 
enjoys  increasing  visibility  not  only  in  Alberta  or  Canada  but  also  among  policy  makers 
in  other  nations.  AISFs  continued  progress  within  one  of  the  world's  very  highest 
performing  systems  will  be  keenly  observed  by  scholars  of  educational  change  and  policy 
makers  focused  on  improving  student  learning  from  around  the  globe. 
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